Plotting Shapefile Data Using Geopandas, Bokeh and Streamlit in Python.

I was recently introduced to geospatial data in python. It’s represented in .shp files, in the same way any other form of data is represented in say .csv files. However, each line in a .shp file corresponds to either a polygon, a line, or a point. A polygon can represents certain shapes, so in the given context of maps and geospatial data, a polygon could act as a country, or perhaps an ocean and so on. Lines are used to represent boundaries, roads, railway lines etc. Points are used to represent cities, landmarks, features of interest etc. This was very much new to me, so I found it fascinating to see how any sort of map can be broken down into polygons, lines, and points then played around with using code. I was also introduced to streamlit, which provides, say, an alternative to Jupyter Notebooks but with more interaction and in my opinion, better visual appeal. I think one distinct advantage Jupyter Notebook has is compartmentalisation, and how good code and markdown look next to each other, whereas streamlit seems to be more visually appealing. However, one big advantage streamlit has is the fact that it is operated by the command line, making it much more efficient for a person like me who’s very much comfortable with typing out commands and running stuff, rather than dragging my pointer around to click objects.

I used the geopandas library for dealing with shapefile data. It’s incredibly efficient and sort of extends native pandas commands to shapefile data, making everything much easier to work with. One thing I didn’t like was how streamlit didn’t have inbuilt functionality to view GeoDataFrames, so essentially that means I have to output geospatial data using the st.write() method, and that just results in some ugly, green-colored output, very much unlike the clean, tabular output you get when you use st.write() for displaying dataframes. It’s also a bit surprising how st.dataframe() doesn’t extend to a GeoDataFrame, but eh, it works for now.

Bokeh is new for me, I decided to start out with making plots using inbuilt geopandas and matplotlib functionality rather than move straight to Bokeh. Hence, in this post, I’ll be going through how I made and annotated some maps using geopandas, then extended that to Bokeh to make my code much more efficient. A huge advantage Bokeh brings to the table is that it can be used to store plots, so no going back to earlier cells to find a plot. Just output it to an html file and write some code to save the contents of that html file, you’re then good to go.

The very first thing I did was create a very basic plot showing Germany filled with a blue color. Here’s a small piece of code that accomplishes that.

Plotting Germany

This simple takes the geodata for the entire world, in low resolution. I then select the data specific to Germany, plot it using geopandas, turn plot axes off just to make it look better visually, and output a plot to my streamlit notebook (do I call it that?). Here’s the result

As you can see, it’s a very simple plot showing the nation state of Germany. Now, I thought I’d extend this, make it look better, and annotate it with the names of some cities and the capital, Berlin. The very first thing to do here is to get some data for each city, their latitudes and longitudes, to be specific. You can either import that via a shapefile, which will have each location as a point. However, I manually inputted 6 cities and their data from an internet source to a dataframe, and then used that to annotate my figure, which I’ll be talking about now.

The dataframe at the top contains my city position data. Down below, I’m creating another GeoDataFrame that holds each cities data as a point object. You’ll notice that I’ve used the points_from_xy() method while creating the city dataframe. points_from_xy() wraps around the point() method. You can view it as equivalent to [point(x,y) for x, y in zip(df.Longitude, df.Latitude]. It works really well and removes the need to have a for loop, making, making my code much more efficient. I’ve then plotted the same map as above, except with a white fill and a black outline (better looking imo). After that, I’ve gone over each point using a for loop and added a label, which is the City name (stored as just “Name”). I’ve also increased the size of the marker for Berlin, given that its the capital. The last step is just adding a red marker to indicate each city’s position. Note that st.pyplot() is a streamlit method that outputs any figures we might have. Here is the output of the code above.

I think this looks much better.

Now, I decided to plot a map showing all the main railway lines in India on top of a blank map of India, and output this to a new .html page as a bokeh plot.

As you can see the, the code for this is very simple. I’ve firstly plotted the map of India using the get_path functionality in geopandas. Then, for the sake of visbility, axes lines have been turned off. Then, I’ve read the railway shapefiles, which consists entirely of line objects, and plotted it using the pandas_bokeh library, outputting to an html file. Here’s the result

I find working with geospatial data to be terribly useful, and very much exciting. I think that’s partly because I love data science; playing around with data and modelling it, and working with geospatial data opens up an entire realm of possibilities. I’d describe the experience of making my first shapefile plot as something akin to working for the first time with time series data. Much like time, it adds another dimension to what one can do. In the coming weeks, I’ll be blogging more often, hopefully once every week, on geospatial data specifically. The contents of this post are not even an introduction to what can be accomplished using geospatial data. Next point of exploration, for me, is going to be how to depict elevations, and use color density to indicate population density, average cost of living, gdp etc. One immediate application that comes to mind is making a map that uses color to reflect how much covid-19 has impacted a country, based on not just cumulative confirmed cases, but also factors like healthcare expenditure, economic downturn, unemployment etc. I think it’ll be interesting.

You can find the entire code here.

Nihilism and Importance.

At the start, we humans thought we were the center of the universe. How naive. We thought this lie to be a universal truth that granted us special status in the universe, we were at the center of God’s creation, and thus the most prized of his inventions. Then, the heliocentric model placed the Sun at the center of the universe. In what was a huge blow to human pride, we came out as self-appointed winners on the basis of us still being God’s cherished inventions. We then learned that the Sun wasn’t the center of the universe, but instead the center of one of trillions of Solar Systems in constant revolution around the Milky Way center. Then, we found out that we were just another planet around another star in another galaxy that was one of trillions upon trillions. Now, there is a fair chance that we may just be one of quadrillions of universes.

While our insignificance in the cosmic scheme of things has nought but grown, we’ve still found ways to make ourselves feel unnecessarily special. The first of these is the proclamation that we’re the only known form of intelligent life in the known universe. Now that might hold true what of what importance is being intelligent when regardless of its existence, the horsehead nebula still remains untraveled, the TRAPPIST system remains unexplored, and the flat earth theory still remains believable to an absurd number of people. We’re intelligent relative to the natural life on Earth (although maybe not, octopuses display far more intelligent behavior than us). On a cosmic scale, the human species is to the universe what the the flutter of a butterfly’s wing is to storms on Jupiter, irrelevant. In fact, the argument that being intelligent is being special is in itself flawed; an intelligent species wouldn’t knowingly harm its ecosystem and put politics above science.

So, one may ask, what does make us significant? Is it love? friendship? Science? The former two make individual lives important, but not an entire civilization. Love isn’t a metric for progress, scientific progress is, and its measured by the Kardashev scale. The Kardashev scale measures a species’s technological progress. I denotes optimal utilization of all planetary resources. II denotes optimal utilization of host star’s energy output. III denotes optimal utliization of your entire galaxy’s energy resources. We haven’t even reached point I yet, and there’s no saying we ever will given the way we’ve used our planet’s resources is destroying it. The only way to become significant in a cosmic sense is to expand our barriers and become a space-faring civilization. Our solar system provides a variety of environments, everything from the hellish landscape of Venus to the subsurface oceans of Europa, populating them is both an engineering challenge and a stepping stone to going beyond.

Having a measurable impact requires scientific progress, but humans would rather continue playing tic-tac-toe rather than switch to Minecraft. Rather than adopt incredible challenges in the form of navigating the harsh environment of space and settling humans on other bodies, our politicians choose to fight and debate and argue endlessly. That’s because the entire concept of modern politics is somewhat flawed. Politicians can’t make decisions that balance on the scale of decades, even centuries. They cannot comprehend the importance of science, because anyone who is mildly scientific would never go into politics, and fixate themselves on which country did what and their endless agendas rather than the one big agenda that should occupy their mind: what is our place in the universe?

And it goes without saying that space offers more economic potential than anything else, so the entire notion of immense space exploration budgets drawing money away from the economy and facets like healthcare and education is founded in quicksand. Asteroid mining offers returns on the level of trillions of dollars, if not more. Lunar colonization offers dramatic reductions in the cost of space exploration and a gateway to other bodies in the Solar System. Europa and offer the chance to find life beyond Earth, or at least understand how modern Earth came to be. Space offers a dizzying array of possibilities, but we ignore them all.

As the Ancient one put it, we’re a “momentary speck within an indifferent universe,” having an impact requires understanding of just how important technology and science are.

But then again, do people really get that any longer? Ancient humans had access to the universe via a tool that is ironically useless today, the naked eye. Looking up offered a window into a larger world, unlike today where the night sky is pitch black like in all of Enid Blyton’s stories. This sparked curiosity, it led to the development of calendars and timekeeping and birthed the concept of studying, which has historically been awesome. Even astrology came into being, one of our first attempts to explain how the universe impacts us. We’ve lost sight of that burning curiosity now. People think they have access to a whole new world through their smartphones, but no one realises what we’ve lost along the way. We’ve lost the reason why we began being intelligent: the night sky.

Solving the UEFA Champions League problem on Code Chef with OOP

So I decided to try out code chef, and chose to solve the UCL problem listed on the home page of the ‘practice’ portion of code chef. The problem statement was detailed and easy to understand, you can view it here. It basically involves inputting the match results for a league in the following format:

<home team> <home team goals> vs. <away team goals> <away team>

for instance, fcbarca 5 vs. 1 realmadrid

12 match results are inputted, and match results for T leagues are taken in. The output for each league should be the names of the winning team and the runner-up. For instance, the output for the following league schedule/results,

manutd 8 vs. 2 arsenal
lyon 1 vs. 2 manutd
fcbarca 0 vs. 0 lyon
fcbarca 5 vs. 1 arsenal
manutd 3 vs. 1 fcbarca
arsenal 6 vs. 0 lyon
arsenal 0 vs. 0 manutd
manutd 4 vs. 2 lyon
arsenal 2 vs. 2 fcbarca
lyon 0 vs. 3 fcbarca
lyon 1 vs. 0 arsenal
fcbarca 0 vs. 1 manutd

would be

manutd fcbarca

Here are some basic rules we observe for football tournaments. Each victory gives you 3 points, each lose gives you 0 points, each draw gives you 1 point. If two teams have the same point tally, the team with the higher goal difference is placed first. For simplicity’s sake, we’re assuming here that no two teams with the same point total will have the same goal difference.

I defined two classes, League, and Team right at the start. The only attribute an instance of the League would have would be table, which is a dictionary wherein the key is a string pertaining to the team’s name and the value if a Team object containing all necessary information. I also put five methods inside the League class, which I’ll be explaining just in a bit.

League Class

The does_team_exist() method just returns true or false values pertaining to whether or not the inputted string is a key in the dictionary field. In my main code, if does_teaam_exist() returns False, the add_team() method is called and the self.table attribute is updated with a Team object being added to it. If does_team_exist() returns True, the program continues. match_won() and match_lost() are pretty well defined. The former calls the win() method for the Team object corresponding to the Team that won and the lose() method for the Team object corresponding to the Team that lost. You’ll notice that we also pass in the goal difference here. The latter calls the same draw() method for both, and no gd is passed in because it equals 0 for both times in the event of a drawn match.

return_top-two() is our most important method. It creates a list of all values in the table dictionary. Then, I used a lambda function to sort on the basis of two parameters, points and goal difference. You’ll notice that my first element in the lambda function inside the tuple is the number of points associated with that object, and the second is the goal difference for that team. After sorting all Team objects, I return the team names of the last and second last objects.

Now, I’ll show you the Team Class.

Team Class

This is much simpler. When initializing, the only argumentative input is the team name, while points and team difference are initialized to 0. The win() method increases points by 3 and goal difference by the argumentative input. the draw() method only increments points by 1, and the lose() method decrements goal difference by gd. This is followed by three get methods, two of which are used in the lambda function shown above, and one which is used to get the desired output for our program.

Now, the only part left is our main class or working code. Here it is.


We take in T as our first input as defined by the problem statement, then iterate over each League. Before going into the results for each league, I’ve initialized league to an instance of the League class. I then iterate over each of the 12 fixtures in each league. Splitting the input and extracting certain indices (see code) gives us our values for each team’s name and the number of goals it scored. You can see that I have two “if not”s up there that initialize a new team object inside the table attribute of the League class if the team in question is not there already. Then, I compare home team goals and away team goals and call the appropriate method where needed. Our output is the value returned by the return_top_two().

You can check out the full code here.

Credits to code chef for the problem statement.

UI to mail NASA APOD image to inputted email [#2 NASA APOD Series]

I’ll be going through using Flask to create a 2-webpage website with a simple form on the first page that takes in your email, clicking the submit button sends an image of the latest NASA APOD image to the inputted email. For this, I’m using the script written and explained in the last post in this series, which can be found here. So in this post, I’ll be going through the flask templates i created, and my form functionality, which is pretty simple.

The first step is to create a .html page and a file for our website. Here’s the code for the following.

<body> tag of .html file

So the image above shows the code I’ve written for the body tag, which is all that’s worth looking at. The form data returned in the file is used to just show the form and a submit button, very simple functionality so I’m not going to go that much into detail for this part. We only have an email field and a submit button, clicking the submit button firstly checks whether or not the email field has a valid input. If it doesn’t, the field is cleared and a red outline is put on it. If it is, an email containing the latest NASA APOD image is mailed to the inputted email, and the user is redirected to another webpage.

This is the code in my file. I’ve initialised my Flask app with the first line and given it a secret key (for forms). The send_email() function contains the code explained in the previous post. It’s called if and only if the submit button is clicked on the home page and a valid email address is filled out. If these conditions are satisfied, the send_email() function is called with the inputted email as the only argument. The code then redirects the user to the bye.html page, which is just a small html file displaying the words “bye.” I did it for the sake of seeing whether or not the send_email()function has been called inside home.

My functionality for setting up the email form is very simple. I use the wtforms module.. I’ve created emailButton class wherein the the email is set to a StringField object, and submit is set to a SubmitField object. The library provides validator methods. DataRequired() ensures that some data is inputted into the field, and Email() ensures that the given input is a properly formatted email.

This is what our webpage displays. After entering a valid email and clicking submit, here is what I’ve received in my inbox.

Next time I’ll have a more styled layout, and I’ll go through the countdown functionality that will send emails to inputted email addresses until the number of inputted days becomes zero. Project should ideally be complete when that’s done.

You can check out the full code here.

Using Python to email NASA APOD API data to a custom recipient [#1 NASA APOD Series]

I’m doing a mini-project as a learning experience. The first step here involved two sepearate functionalities that I’ve now combined and will explain here. Firstly, I had to write a few lines of code to actually use the API and save the image it returns to the current directory. This was really easy. I then had to learn how to use smtplib to send emails in python. I used this article on Real Python to do so. This post on Stack Overflow was also helpful in instructing me about attaching image files to an email. You can find the entire code in here, on my GitHub. I’ve been having some issues with WordPress plugins, so I’ve just attached screenshots of my code for this post.

right, so the first thing we’ll be looking at is accessing the NASA APOD API and storing the latest NASA Astronomy Picture of the Day Image in the current directory.

I firstly import the necessary modules. After that, we’ll access the API with our key. The url contains the link given in the offficial documentation for accessing latest image. The API returns a json object with keys pointing to a description of the image, a hd url, and a nominal quality url. I’ve used the urllib module to fetch and store the image to our current repo (stores to which repo we are in or we have changed to with os). Now comes the hard part.

You can take a look at all the modules I’ve imported in the GitHub link i’ve posted above.

Above, we start our smtp_server and initialize our port. I’ve used 587 because I’m using the TLS method, I think you’ll need to use 465 if you use the other method. I’ll be using to send this email. This is a throwaway account I created with all security features turned off. I’m mailing it to, which is one of the emails I use. We then input the password for the email we will be sending it from. Then, we’ll our message will be set to an instance of the MIMEMultipart objective with “alternative” as our input parameter. We then key in some values, like the email to use, the email to mail it to, and the Subject of our email. I’ve then specified html and plaintext messages for the email to contain.

I’ve attached both html and plaintext messages to our email. The procedure for attaching a .png file as an email attachmetn is very simple. Using the inbuilt open subroutine in python, we read the image. Then, we create an instance of the MIMEImage object where we key in the img_data we just read and input the path of our image file. os.path.basename returns your current path, kinda like the pythonic way of writing pwd.

A SSL context is created after that. Check out this Stack Overflow post to learn more about what an SSL context is. We then move into a try except condition, so that we can catch any errors that come our way. A server is established, and server.ehlo() is basically the equivalent of waving hi to your friend across the room. We then start tls, which is a method of encryption for safe and secure communication across the internet. Finally, as a method of the server instance, I send the email to the recipient. The finally block quits the server regardless of whether or not the email was sent and whether or not an error was thrown.

I’ll now create a webpage where you can input your email, and a python script sends the current NASA APOD image to your email. So stay hooked for #2!

Also, FYI, here’s what the email looks like:

Conducting an online MUN and Presiding over the SC.

This weekend, I had the opportunity of presiding over a simulation of the Security Council and being Secretary General of our school’s and my city’s first ever online MUN. I made a commitment to organizing a conference way back in August, at the start of my tenure as MUN Club President. COVID-19 provided me with a unique opportunity, the chance to conduct a virtual conference unlike other MUNs in how effective I wanted it to be and the fact that all committees would be working together to come up with solutions for the current pandemic. We had crisis updates in the General Assembly impacting proceedings in the Security Council, speeches PM Modi being broadcast in the GA, and the Security Council and WHO having a joint crisis on the basis of Iranian bioweapons allegedly supplied by China. It was truly a wonderful experience.

I had the idea of conducting such a conference over a month ago. I immediately reached out to my peers and teachers, and got the planning underway. We made a team, divided all the work, and got everything done very quickly. In fact, the hard work each team member put into the event, whether it was our logistics heads texting every single delegate, or our Vice-chairs who were tasked with working on Background guides, is commendable. Now, more about what I did.

As President of a Security Council meeting that was investigating COVID-19’s origins and evaluating the possibility of it being a bio-weapon, the fact that it was a simulation did not draw away from how serious the situation was. Debate started with the Russian Federation introducing a verified, reliable research paper that clearly proved how COVID-19 couldn’t be artificial. I then urged our committee to investigate some more pressing prospects such as COVID-19 being a natural outbreak intentionally allowed to get out of hand and it being a sort of evolved form of SARS for which a vaccine already existed (concealed by the Chinese). After a round of statement speeches, I introduced the first crisis of the day, a video of a worker in the Wuhan virological center stating how the virus was laboratory made and intentionally released. Not soon after, one of our MUN mentors who has now graduated high school joined as the guest delegate of Japan, accused China of approaching him in informal session with an offer to trade the formula for the vaccine for economic and financial support, and abruptly left; committee had just livened up. Allegations flew across the floor during formal session, and insults during informal session. China became a sort of punching bag for the rest of the committee, soaking up countless jibes and taunts and resorting to reminding the other delegates about what I said and the source Russia provided.

I then took it a step further by releasing another video made by a resident in Wuhan claiming they were receiving ‘magic medicine’ from the authorities, and that all who refused to take it were being welded shut into their homes. This effectively meant that the Chinese had a vaccine, or had discovered an accidental cure. In either case, it meant China was concealing something important from the rest of the committee. This marked the end of the first day. However, at midnight, the General Assembly convened for a midnight crisis (a fascinating aspect of MUNing I saw happen for the first time), and witnessed PM Modi deliver a speech on how an Indian lab had created the vaccine. We then received an informal tip-off hinting upon a US-India partnership involving the US and India distributing the vaccine to their civilian populations before the rest of the world, and reaping the economic benefits of selling a highly demanded product to 189 countries at a very high price. I took this opportunity to text all members of my committee with a midnight update, alerting them to what had just transpired.

Committee convened again at 8 am in the morning. We debated on the draft resolution at hand for the first two sessions, cut away more than 80% of it and ultimately vetoed it, so 5 hours of effort went down the drain, not because we failed to pass a resolution, but because I failed to engage delegates in the resolution drafting process. Judging the results of an informal vote and what I believed to be best for committee, I decided to have a series of crisis updates for the last two sessions.

The first of these was an announcement by the Indian chief of army staff, detailing how the sole laboratory with the cure for COVID-19 had been blown up via a terrorist attack, and that the delegate of India to the Security Council had been assassinated. Dealing with the fact that the only verifiable vaccine was gone, and that China had a potential vaccine, all blocs split apart as China, Russia, and the US faced a series of allegations. This was followed by an announcement by the Russian Federation detailing how they had obtained the vaccine and were prepared to sell it to all other countries at subsidized prices, but would mass produce it in the public sector and not distribute the formula so as to reap colossal economic benefit.

So, that’s how committee ended. While we certainly didn’t have any progress towards satisfying our agendas, we had that kind of accusation-abundant, intense debate where blocs don’t really matter that brought out the best in our delegates. It was incredibly interesting to see our delegates handle incredibly absurd and riveting crisis updates, and do so while coping with incredibly arcane unmods. Conducting eJPISMUN was a brilliant experience, and the way our conference turned out exceeded the aspirations I had for it when I came up with the entire idea.

What is Intelligence?

Reading Homo Deus by Yuval Harari really made me think about some stuff. Here’s what I concluded about the nature of intelligence:

I’m going to go out on a limb and say drawing conclusions from a given amount of data can be defined as a reasonable metric for intelligence. For instance, in math classrooms the most intelligent students are those who can work through problems with ease and in the absence of exterior help. However, machines can do the same in a matter of nanoseconds. In fact, machines have already replaced humans in terms of calculation, with pocket calculators now a classroom tool and supercomputers enabling us to explore much larger numbers like the googolplex. Now, machine learning is replacing humans in the broad area of problem solving. Artificial Intelligence is being used to diagnose patients with various maladies, effectively reducing a doctor’s job from diagnosis and cure to just cure, which may be cut down even more due to the development of robotic surgery tools.

It’s safe to say machines are more intelligent than humans in the modern world. Machines are already pioneering and establishing new frontiers for humankind. From Mars, where mechanical probes are more commonplace than humans, to the Marianas Trench, where remotely piloted submarines have been before us. The advent of artificial intelligence will seal what is surely the intellectual dominion of computers over humankind. This begs a very important question, what makes us unique?

The first two candidates that come to mind are curiosity and sentience. On a side note, Octopuses display both those traits. With a separate brain in each of their eight tentacles, a camouflaging ability that puts chameleons to shame, and a profound tendency to form emotional bonds with everyone they meet and memorise them. Octopuses are to us what velociraptors were to Dr Alan Grant in Jurassic Park; thank god they dwell in the ocean and not on the surface.

Another pressing question isn’t whether or not sentience makes us unique, but whether or not it makes us weak. Machines lack emotions, most people call that a drawback, but is it? Imagine this scenario:

It’s the year 2050, and humankind has a noticeable presence on Mars, with upto ten thousand people calling themselves ‘Martians.’ A disease outbreak the likes of which have never been seen before occurs, killing every single Martian. A ship departs from Mars, bound for Earth. Now, a human gateway operator in a space station orbiting Earth has two alternatives, destroy the ship from Mars using a nuclear warhead to prevent a potential outbreak on Earth, or let it pass and risk an extinction-level event. Here’s a twist, his wife and kids are aboard the ship. This creates a situation where there’s room for doubt, which leads to potential for catastrophe, and when a few lives come in the way of all life, catastrophe does occur. Now imagine a benevolent AI operating the space station. Weighing the

lives of those aboard the ship against life on planet Earth, it would pull the trigger and hence end the risk of another outbreak.

Movies and books about future civilizations and artificial intelligence always contain fascinating plot twists where all of a sudden love emerges as the differentiator between humans and machines. Machines go haywire, and the humans win. Interstellar ends with love ‘transcending reality,’ and being the one ‘force’ that saves the entire planet from starving. Why? Because a father loved his daughter very much. Imagine an artificial intelligence program in place of Darth Vader, would it have given in to fatherly love, killed the Emperor, and ensured the victory of the light side in ​The Return of the Jedi?​

What makes machines efficient is their lack of emotional judgement. Did Curiosity, the rover, ever feel lonely on a planet from which Earth is visible as nothing more than a speck? No, it didn’t. Computers’ lack of sentience and their astounding computational and logical powers make them dominant over us.

Implementing Univariate Linear Regression in Python

The objective of this post is to explain the steps I took to implementing univariate linear regression in Python. Do note that I’m not using libraries with inbuilt ML models like sklearn and sci-py here.

Here is our gradient descent function, utilising mean squared error as the cost function.

def gradientDescent(theta, alpha, iterations):
    m = ex1data.shape[0]  # finding the number of trial examples
    n = ex1data.shape[1]  # finding the number of features + 1
    for iteration in range(iterations):
        total0 = 0
        total1 = 0
        for row in range(m): # iterating over each training example
            hypothesis = 0
            for val in range(n-1):
                hypothesis += ex1data.values[row][val] * theta[val][0] 
            load = hypothesis - ex1data.values[row][n-1]
            total0 += load*ex1data.values[row][0]
            total1 += load*ex1data.values[row][1]
        temp0 = theta[0][0] - ((alpha*total0)/m)
        temp1 = theta[1][0] - ((alpha*total1)/m)
        theta = [[round(temp0, 4)], [round(temp1, 4)]]
    return theta

We carry out gradient descent 1500 times here, by setting iterations equal to 1500. Our starting values of theta are 0. Now, for those of you who don’t know how gradient descent works, here’s a short explanation that attempts to cover the crux of it. Intuitively, we subtract each of our output values as given by the hypothesis function by the target value we’re trying to predict, then square the difference. The gradient descent update rule subtracts the partial derivative of this (beyond us mortals for now) from the existing values of theta – updating them. This entire process repeats 1500 times until gradient descent converges to an optimal value of theta, or the minimum point of the cost function.

Now, we plot our data to see what it looks like.

m = ex1data.shape[0]
print ('No. of training examples --> {}'.format(m)) # outputting the number of traning examples for the user
eye = []

for i in range(0,m): 
    eye.append(1)  # creating an array of 1s and adding it to X
if len(ex1data.columns) == 2:  # to avoid an error wherein ex1data already has the column of 1s
    ex1data.insert(0, "feature1", eye)
print ('here is theta (initial)')
theta = [[0], [0]]

Now, we firstly add a column vector consisting entirely of 1s, of dimensions m by 1, to our feature matrix. Hence, we have a feature matrix of m by 2, where one column pertains to our variable data and another to a column vector of 1s. We then initialise both values of theta to 0. Our learning rate, alpha, will be set to 0.01 for gradient descent, and we will execute gradient descent 1500 times. After running gradient descent and plotting our predicted values against the actual dataset, this is what we get:

Pretty cool, right?

This entire example was based on solving the Week 1 problem set of Andrew Ng’s machine learning course on through Python. So credits to Stanford University. Stay quarantined, stay safe!

Summary of Reading – March ’20

Homo Deus, Yuval Noah Harari – Scary, impactful, dire, and exciting. As Harari clearly stated towards the end, the book outlines probabilities and not prophecies for the future, but each advancement in technology and each step forward in dataism he predicts bifurcates into multiple, significant impacts on us, humankind. The book pens down paths towards everything from extinction events akin to humans wiping out animal populations, to a downfall of liberalism and a completely deskilled society. Being a practitioner of data science, Harari’s message was something that just didn’t have deep implications, but also seemed real enough to be taken seriously. Another thing that stood out for me is Harari’s explanation of capitalism as a distributed data processing system, and how no single person knows the great machine works, but the great machine keeps on working; how politicians are genuinely oblivious to the working of their nations, because the working of their nation is dependant on how everything falls together and how social forces interact. I compared this to the Foundation Saga as well, where individuals play no role in the greater scheme of things, but what Harari predicts is of a much larger magnitude: humans will play no role at all.

Astrophysics for People in a Hurry, Neil deGrasse Tyson – Worth a read. I had a decent idea about the stuff Tyson talks about in this book beforehand, but that didn’t prevent it from being enjoyable. I especially loved the descriptions of various elements in the cosmic sense: where they came from and their role within the universe. His entire commentary on dark matter and dark energy was also catching, especially how the effect of both counter-acts Einstein’s cosmological constant, which I’ve never heard of before. I especially remember Tyson pointing out how Einstein’s greatest blunder wasn’t the cosmological constant, but instead labelling the cosmological constant as his greatest blunder. He doesn’t betray the purpose of his book, delivering the wonder of the cosmos to all audiences, which is unique given how most books of the non-fiction, science genre pretend to be easy to grasp for anyone at all but end up being warped and confusing for most.

Taking a look at the past and the future of Mechanical Engineering.

Arguably humankind’s greatest engineering and technological accomplishment, the Apollo Program landed a human on a body other than planet Earth for the first time. Constructing and using the command and lunar modules was a major engineering feat. As per the Lunar Rendezvous method, the command module remained in orbit (with one occupant), and the lunar module went down and came back up (carrying two astronauts) – an elegant solution. Making the lunar module was a humongous challenge; it needed to be as lightweight as possible without compromising on any of its functions. In the end, NASA engineers ended up removing the seats themselves.

The construction of the Saturn V rocket is another impressive landmark. It burned 15 tons of fuel per second at lift-off, generated 7.5 million tons of thrust, and stood as tall as a third of the Empire State Building.

Personally, I find how NASA used archaic computer systems to process data and monitor the mission, doing everything from calculating trajectories and ensuring constant communication between Houston and the Eagle, pretty awesome. Do note that these computer systems, today, would be less powerful than an Apple Watch.

However, the most significant impact of the Apollo Program deviates hugely from the engineering and technological advancements it made. It showed Humankind an image of the Earth as seen from the Moon. Seeing this pale blue dot from afar, so small that it could be covered with the back of your thumb, had a profound, overbearing impact on people.

Concerning the future, I think the greatest feat of engineering in the next 100 years will be constructing a human colony on planet Mars. Doing so will require the development of means to transport thousands of tons of equipment, materials, and resources (along with humans) to Mars. We’ll need to set up basic infrastructure on Mars, and build habitats/shelters that enable humans to survive the harsh environment of the Red Planet; a thin atmosphere, dangerously high radiation levels, and wildly fluctuating temperatures.

Transporting materials and humans there will require space travel infrastructure. The sheer amount of traffic involved could warrant the construction of the first spaceport outside of planet Earth, on the Moon, from where taking off would be significantly easier due to a weaker gravitational pull. Reusable rockets will be key, so they must endure trips of upto 2 years upto ten times throughout their lifespans.

Providing water, food, and breathable air to future Martians will require technological innovation. Lab-based agriculture will not not be enough to feed an entire population, so alternate sources of food must be developed. Habitats would need to counter wildly fluctuating temperatures by cooling the system during the night, and heating the system during the day. Furthermore, setting up energy infrastructure on Mars will be even more hard. Solar is a viable option, but is rendered ineffective during dust storms. The current alternative, in my opinion, is nuclear fusion, given that the 100 years of progress that take us to a position where we can construct a colony on Mars also give us the means to harness the awesome power of nuclear fusion.