Locking and Unlocking .pdf files

Here, I’ll be showing you how to set the same password for all .pdf files in a directory simultaneously using python, instead of manually setting passwords for each pdf. This comes in handy for files with over hundreds of .pdf documents. The full code for setting passwords can be found here, and the same for unlocking files is available here.

We will be using the PyPDF2 and os modules. To start off, we’ll get the user to input their password and root directory. Then, we’ll use the os.walk()function to iterate over each file in that directory and its sub-directories.

for (root, dirs, files) in os.walk(path):
	for filename in files:
		if filename.endswith('.pdf'):
		    pdfFile = open(root + '/' + filename, 'rb')
			pdfReader = pypdf2.PdfFileReader(pdfFile)

Above, we store the result of each ‘walk’ as a three-element tuple. We then access each file in the current directory, and use an if statement to check whether or not the file concerned has a .pdf extension. Then, we open the file in a read binary format, and also initialise a pdfFileReader object for the file.

Now, to make an encrypted pdf using PyPDF2, we’ll read one file and copy over its contents into another file, then encrypt the second file and delete the first. Do note that opening the first file in a write binary format won’t delete it, it’ll delete the data inside the file, but not the actual file. To delete files, we can use the os.remove() function.

# inside 'endswith('.pdf')' if

if pdfReader.isEncrypted == False:
    pdfWriter = pypdf2.PdfFileWriter()
    for pageNum in range(pdfReader.numPages):
		pdfWriter.addPage(pdfReader.getPage(pageNum))

	pdfWriter.encrypt(password)

	if filename.endswith('decrypted.pdf'):
		resultPdf = open(root + '/' + filename[:-13] + 'encrypted.pdf', 'wb')
	else:
		resultPdf = open(root + '/' + filename[:-4] + 'encrypted.pdf', 'wb')

	pdfWriter.write(resultPdf)
	os.remove(root + '/' + filename)
	resultPdf.close()
	pdfFile.close()
	

Now, we firstly check whether the file in question is encrypted or not. If not, we initialise a PdfFileWriter object, and copy over the contents of the file to the object. Using the user-inputted password, we encrypt the new file and then save it with a suffix of ‘_encrypted.pdf.’ Then, the os.remove function is used to delete the starting file.

Unlocking files has the same methodology. Using a python program also saves a lot of time when say unlocking hundreds of files. Once again, we use os.walk() to traverse our root directory. Here, we also check for whether the file is encrypted, and proceed only if it is.

if pdfReader.decrypt(password) == 1:
    pdfWriter = pypdf2.PdfFileWriter()
    for pageNum in range(pdfReader.numPages):
        pdfWriter.addPage(pdfReader.getPage(pageNum))
	
	if filename.endswith('encrypted.pdf'):
	    resultPdf = open(root + '/' + filename[:-13] + 'decrypted.pdf', 'wb')
	else:
	    resultPdf = open(root + '/' + filename[:-4] + 'decrypted.pdf', 'wb')
	pdfWriter.write(resultPdf)
	os.remove(root + '/' + filename)
	resultPdf.close()
	pdfFile.close()
else:
    print('password incorrect for {}'.format(root + '/' + filename))
    

If the inputted password works for a file, the file is unlocked, then its contents are copied over into a new file with no password. Then, the encrypted file is deleted, and the new file replaces it. If the inputted password is incorrect, the program outputs the name of the file.

Application-wise, these programs are very efficient. For example, if there is a website with hundreds of free resources (many of which are password protected PDFs), you can download all these files and use the program above to unlock all files simultaneously. Using a brute-force attack (coming soon), we can also download hundreds of files and try to hack them using a spin-off of the program above.

Using Python to make Multiplication Tables in Excel

A really easy problem. Here’s the full code.

Now, we need to create an n by n multiplication table in excel using a python program, where n is an arbitrary, positive integer. We will do this using the openpyxl module. Firstly, we’ll initialise our notebook and set our current sheet to Sheet, which is the default active sheet.

wb = openpyxl.Workbook()
sheet = wb['Sheet'] # or wb.active
n = int(input('enter n| '))

For the purposes, a ‘bold’ font style object has also been created. I will assign this to the row and column headings later.

for i in range(2, 2+n):
	sheet['A' + str(i)].value = i-1
	sheet['A' + str(i)].font = boldFont
	sheet[get_column_letter(i) + '1'].value = i-1
	sheet[get_column_letter(i) + '1'].font = boldFont

Above, all I’ve done is create row and column headings for all integers upto and including n, with each heading being written in bold characters. Now, time to create the table.

for i in range(2, sheet.max_row + 1):
	for j in range(2, sheet.max_column+1):
		sheet[get_column_letter(j) + str(i)].value = sheet[get_column_letter(j) \ 
		        + '1'].value * sheet['A' + str(i)].value

Making the table is really simple. What I’ve done is iterate over each row heading, and then over each column heading inside the row, then find the product of these two and assign it to the required cell. You could also do this using excel formulas, but I think this solution is reasonably efficient.

Here’s another solution that I found on github.

for rowNum in range(1, number+2):
	for colNum in range(1, number+2):
		if rowNum==1 and colNum==1:
			sheet.cell(row=rowNum, column=colNum).value=''
		elif rowNum==1:
			sheet.cell(row=rowNum, column=colNum).value = colNum-1
			sheet.cell(row=rowNum, column=colNum).font = boldFont
		elif colNum==1:
			sheet.cell(row=rowNum, column=colNum).value=rowNum-1
			sheet.cell(row=rowNum, column=colNum).font = boldFont
		else:
			sheet.cell(row=rowNum, column=colNum).value = (rowNum-1)*(colNum-1)

It is shorter, but I don’t think it’s that elegant due to the usage of if/else statements. But that’s just personal bias. Instead of iterating over headings and content cells differently, the author here iterates every single cell (including 0,0) in one for loop.

Here’s a sample multiplication table for n = 18:

Arduino Crystal Ball

This is the 11th project in the Arduino Starter Kit. Do note that I have skipped the 9th and 10th projects because my DC motor isn’t working.

Setup

Here’s what we need:

  • Arduino UNO + Breadboard
  • Jumper wires
  • LCD Screen
  • Potentiometer
  • Tilt Sensor
  • 1 x 10 kΩ Resistor
  • 1 x 220 Ω Resistor

Layout

Here’s our schematic, as per the Arduino Projects Book.

And here’s my incredibly messy build:

Code

We will be using the LCD Display (Arduino) library to control our screen. While setting up, all we need to do is print out the initial message we want the user to see.

void setup() {
 lcd.begin(16, 2);
 pinMode(switchPin, INPUT);
 lcd.print("Crystal Ball");
 lcd.setCursor(0, 1);
 lcd.print("At your service");
}

Then, using the random() method, we will give our output.

reply = random(8);
switch(reply) {
    case 0:
    lcd.print("Yes");
    break;
    case 1:
    lcd.print("Probabale");
    break;
    case 2:
    // ...

That’s all there is to it. You can see the full code here.

Metrics for Analyzing Urban Societies

If we want to analyze specific regions or communities within cities, there must be some indicators that act as differentiators. For example, carbon dioxide concentrations, literacy rates, water hardness levels, diabetes levels etc. We can use multiple quantitative metrics. However, the hard part is finding relationships between these metrics and interpreting them.

With regards to Jaipur, there are various ways to collect data and use metrics. The first, simpler method is to set up your own grid of sensors around the city, each sending data in files based on its latitude and longitude rather than identification number. The second would be to get in touch with say the state government, or a sufficiently large private company.

Now, along with collecting numerical data, we will also need some qualitative data (and assign classes) to establish patterns. For example, if a specific region with a certain pincode has an abnormal concentration of carbon dioxide and a high usage of cigarettes, we would expect severe health, or even comfort problems there.

Getting the data is the hard part. Kaggle has a database on Jaipur, but only regarding weather data, which sucks. There are various other governmental sources, but most are written in hindi and are consequently very hard to input into a model as a language to language model/framework will be required. If all the sheets have a standard template, we could type in the data manually for the first few sheets, and then figure out a scraper to do the job for us.

Summary of Reading – August 19′

The Glass Palace; Amitav Ghosh – A book reliant on emotions and relationships. Really didn’t bode well with me. The end was depressing and cruel, with characters having been developed throughout the book being separated and consequently dying. I usually don’t read books in this category. This book just ensured that i’ll continue to do so.

The Hitchhiker’s Guide to the Galaxy; Douglas Adams – One of the best books I’ve ever read in my life. Challenges the borders of imagination. The one thing that stands out throughout is just how absurd, and surprisingly realistic everything seems to be. Adams’s use of humour and sarcasm is also a defining aspect of the book. From mice being ‘pan-dimensional’ beings to depressed robots with brains the size of the planets, this book is one hell of a read for those who love what lies beyond. His little digs at some of our defining aspects also avoid failing to grab attention. There were two that I distinctly remember. The first being the one about construction crews demolishing homes, inflated to gargantuan levels by starting off the book with an intergalactic construction crew that demolishes the Earth (the dolphins knew!). The second is much more subtle – Dent’s use of various ‘stylistic devices’ to provide a feedback for Vogon poetry.

The Restaurant at the End of the Universe; Douglas Adams – This book takes a more poignant tone, but not without degrading the humour. Zaphod entire argument of anyone wanting to be a president not being a good fit for president comes true, dramatically, with the ‘man who controls the universe.’ However, Ford and Arthur’s stay on prehistoric earth grabbed the headlines for me. I adored how Adams manages to use the Golgafrinchans to convey both humor and anger at us humans. Their insane amounts of stupidity, which was responsible for tree-dwellers dying out and being named cave-men, and war being declared on regions with no people did manage to capture the essence of what humans are. Personally, I feel Adams uses this to take a hit at the entire concept of war and egoism.

Life, the Universe and Everything; Douglas Adams – An absolutely brilliant read. Adams hilariously transforms the popular game of cricket into an intergalactic war that kills over 2 ‘grillion’ people. The element of realism makes this worth reading. Personally, I think there were strong connotations between the people of Krikkit and religion. Both behave similarly in the sense that they meet circumstances/events that oppose their ideology or beliefs, and then choose to destroy rather than embrace those self same circumstances. The way Adams uses the white krikkit robots as symbols of death and loyalty (in my opinion) is also spectacular. In all, another stunning book in the series.

So Long, and Thanks for All the Fish; Douglas Adams – This book takes on quite a different tone and setting from the previous books. Surprisingly, Arthur gets a girlfriend. This rounds off his character arc well. He’s settled, got back his planet, and has a person to relax and connect with. It brings out Arthur’s social side, while providing answers to some plaguing questions (but not providing THE question).

Mostly Harmless; Douglas Adams – In typical Douglas Adams fashion, Douglas Adams kills Fenchurch (not exactly kill, but shift Arthur to a universe where Fenchurch doesn’t exist). In addition to being uncannily depressing, it also brings about the destruction of the old guide. Furthermore, to compound this misery, Earth is destroyed yet again, and all of the main characters die. As I said, depressing. It does round off the series pretty well. Adams started by destroying the Earth and ensuring all of the main characters survive, and he ends by destroying the Earth and ensuring all of the main characters die. Perhaps Martin got his inspiration from here.

Arduino Light Theremin (Starter Kit Project #6)

We will use a phototransistor as a light sensor to control a piezo and produce sound – resembling the functionality of an actual theremin.

Setup

Here’s what we need:

  • Arduino UNO + Breadboard
  • Jumper wires/cables
  • 1 10kΩ resistor
  • Piezo
  • 1 phototransistor

Layout

Here’s what the instrument should look like, as per the Arduino projects book.

And here is what I made:

Code

The calibration of the sensor is the most important part in this mini-project.

int sensLow = 1023;
int sensHigh = 0;

void setup() {
    while (millis() < 5000) {
        sensVal = analogRead(A0);
        if (sensVal > sensHigh) {
            sensHigh = sensVal;
        }
        if (sensVal < sensLow) {
            sensLow = sensVal;
        }
    }
}

The millis function returns the amount of time for which our board has been working. Thus, we calibrate for the first five seconds by taking in the minimum and maximum possible sensor input values. These will be used to scale our input values later on.

void loop() {
  sensVal = analogRead(sensPin);
  int pitch = map(sensVal, sensLow, sensHigh, 50, 4000);
  tone(8, pitch, 20);
  delay(10);
}

Here, we take in our input value and map it to the appropriate frequency. Then, using the tone function, we output sound using our piezo (which vibrates as per the frequency provided by tone()). You can play around the frequency and mapping values just for fun.

Arduino Mood Cue (Starter Kit Project #5)

Here, we will turn a potentiometer to consequently rotate a simple servo meter. The arduino will collect the data a

Setup

Here are the components we need:

  • Arduino UNO + Breadboard
  • 2 100 μF Capacitors
  • Jumper wires/cables
  • Potentiometer
  • Servo Motor (with arm)

Layout

Here’s what it should look like, as per the Arduino projects book.

And here is my build:

You might notice that there are some differences between the schematic and what I’ve made. Firstly, the potentiometer is just too large to be placed on the same strip as a capacitor and power/ground connections, which is why I have used orange jumpers to elongate the entire setup. Also note that the capacitors, if connected incorrectly, could potentially explode. I would google up how to connect these and then proceed! It took me a lot of time to find out which pin is the anode and which pin is the cathode. Turns out the pin which has a grey arrow full of zeros pointing downwards is the negatively charged electrode, or the cathode.

Code

You can find the full code here.

#include <Servo.h>
Servo myServo;

To start, we import the library required to interact with the servo motor, and create an instance of the Servo object to handle our motor. Then, all we need is 4 lines of functional code.

void loop() {
    potVal = analogRead(potPin);
    angle = map(potVal, 0, 1023, 0, 179);
    myServo.write(angle);
    delay(15);
}

Firstly, we read our analogue potentiometer value and convert it into an angle for rotating our servo motor. Then, using the servo library, we turn our motor. The delay of 15 milliseconds is to accomodate the time used up in rotating the motor.

Demo

Arduino Color Mixing Lamp (Starter Kit Project #4)

This entire thing is fairly simple. We will use phototransistors to input the RGB components of any light source to our arduino. The arduino will then convert these to digital values, and combine the RGB components to operate a four pin, RGB LED. All we have to do is input to our uno, process the data (fancy word for dividing by 4), and output.

Setup

Here’s what we’ll need:

  • Arduino UNO
  • Breadboard
  • 3 220 Ω resistors, 3 10 kΩ resistors.
  • Jumper wires/cables
  • 4 pin RGB LED
  • RGB color filters
  • 3 phototransistors

Layout

Here’s what the circuit should look like (from the arduino starter kit projects book).

And here is my build:

You can see that I didn’t use the wooden components already in the starter kit to appropriately connect the filters to the phototransistors. This is because those components were too small (probably a defect in the kit), fitting the screens caused them to fly across the room at random intervals.

Code

You can find the full code here. I’m not discussing the setup part here.

void loop() {
    redSensVal = analogRead(redSensPin);
    delay(5);
    greenSensVal = analogRead(greenSensPin);
    delay(5);
    blueSensVal = analogRead(blueSensPin);
}

Here we’re reading in the rgb components of the light source under scrutiny. A delay of five milliseconds has been taken as the arduino takes a while to process analogue inputs and convert them into digital values.

void loop() {
    // ...
    
    redVal = redSensVal/4;
    greenVal = greenSensVal/4;
    blueVal = blueSensVal/4;

    analogWrite(redLEDPin, redVal);
    analogWrite(blueLEDPin, blueVal);
    analogWrite(greenLEDPin, greenVal);
}

Now we divide by four because our analogue input corresponds to digital values between 0 and 1023. However, our output LED requires digital values between 0 and 255. Given that our output LED requires analogue inputs, we use the UNO’s built in DAC to display our combined RGB components.

Note

Many people (including me) initially thought that this entire thing wasn’t working. This isn’t the case. The fact is that the phototransistors that come with the starter kit are incredibly insensitive. They registers values of o,0,0 for room lights (and most lights), which is why there is no discernible output unless one uses a strong flashlight (built-in in smartphones) or a decent light source.

Here’s a small demo. You can visibly see that iphone flashlights have strong blue and green components.

Mini-Demo

Summary of Reading – July 19′

Breakout Nations; Ruchr Sharma; 2012; non-fiction – A fascinating book that is made even more interesting given how it gives a somewhat unique insight into what the world was forecasted to look like almost a decade ago. This is wonderful given how some of the long-term predictions Sharma made in his book are visibly underway, whereas some have been woefully torn apart by economics, politics, and war. Lastly, I especially admired the section on the Indian subcontinent, specifically the portion where Sharma gives his views on the ‘rising’ Rahul Gandhi. A truly wonderful read that links together the past and the present.

Prisoners of Geography; Tim Marshall; 2015; non-fiction – Another brilliant book that managed to convince me that a field of study I previously considered alien and declining plays a much larger role in our world – geopolitics. Marshall successfully navigates the issues of explaining politics using nature. However, on the downside, his language is rugged and not wholly engaging – in stark contrast with the ideas he attempts to portray. There was also too much negativity buried in how most world maps were drawn by ignorant leaders (although I do not know whether it is right to question that).

The Soul of an Octopus; Sy Montgomery; 2015; non-fiction – A fascinating and complex book that enthralled me right from the start. As a presumption, I assumed that, even before starting to read it, that writing about octopuses would be incredibly hard. However, the author manages to weave together some alien facts into a cohesive structure. She also fuses together the best of fiction and non-fiction elements, giving the book a spectral quality. In fact, for a certain period of time, one feels galvanised to make his/her own aquarium. However, what I found lacking was the author’s focus (especially in the latter parts of the book) on worthless stuff that vitiates the flow of language and contradicts her purposes at the beginning. One such example is the chapter on getting octopuses to blind date, which makes no sense to me (given the context).

Contact; Carl Sagan; 1985 – Sagan, once again, doesn’t fail to impress. What I admired about this book was the constant sense of longing, longing to conquer what lies beyond our pale blue dot. Combined with the conflict between humanism and politics which dominates the plot, and the variety of character traits Sagan imbibes into his characters, the book is a stunning read. From people like Kitz, who is frozen in the Cold-War and thus elects to ignore fact and choose hypocrisy, to Hadden, the very definition of a mad scientist x tech wiz, the book contains a wide array of characters differentiated by not just nationality and ideology, but by opinions, thoughts and individuality. Although I fail to agree with Sagan’s final impression of love (cliché) being the only common denominator amongst ‘The Five,’ this book did leave a lasting effect – a will to wonder about whether or not such advanced civilisations actually do exist out there. I completely sympathised with Ellie’s character, and especially her stubbornness and arrogance when it came to denouncing opponents logically. Hadden too had a deep impact on me. His solicitude combined with his love for space and inability to stop believing in the most arcane theories possible made him a role model esque figure. Although his death came a bit too early, I couldn’t help but relate to his desire to see Jupiter’s swirling mass of hydrogen and helium. On the other hand, although I didn’t like it, but one has to accept that Sagan’s repeated emphasis on flawed human characteristics such as political borders transcending into science, inability to believe in the truth and willingness to accept religion did pay off by representing some of the biggest challenges we face as a species. This book will surely leave you shaken, and perhaps cause many to see how important space truly is and how fragile our planet is.


Currently Reading

Automate the Boring Stuff With Python; Al Sweigart; 2015

Programming in Python3; Mark Summerfield; 2015

The Glass Palace; Amitav Ghosh; 2016

Extracting Passwords Using Regular Expressions in Python.

This is my second post on regular expressions. Here, I’m attempting to solve one of the practice tasks given at the end of Chapter 7, Automate The Boring Stuff. Instead of merely identifying whether or not a given string is a strong password, I will write a program that takes a list of strings, and returns a list containing those strings which will make valid ‘strong passwords.’

  • Extract potential strong passwords from text in Clipboard
  • Strong password must have both uppercase and lowercase characters, as well as digits. It must also be at least eight characters long, and must not have any spaces.

So for this piece of code, we will be using ‘lookaheads,’ a new concept for me pertaining to the usage of regexes. Firstly, here is some basic information about ‘lookaheads.’ A ‘lookahead’ is like an if condition. It follows the following format, (note that not many websites refer to it in this way, but I find this intuitive) (?=Regex1)(Regex2). Here, Regex2 is considered if and only if (iff) Regex1 is true. Now, this means that like in an if ... else conditional statement, we can write code that could mean: (?=if digit is present)(read string). Also, do note that Regex1 being true doesn’t affect how Regex2 is read. Another thing we need to do is ensure that the we scan only one password, and not the entire text.

We can do this by either specifying all possible characters except spaces inside square brackets as follows. [a-zA-Z0-9_+$%...]*. Or by simply specifying ‘all non-space characters,’ which is much simpler.[^\s]*. One could also use \w*, but this would not scan characters used commonly in passwords, like %, #, @, ! et cetera.

Then, we need three lookbacks. One to check for a lowercase character. One to check for an uppercase character. One to check for a digit. We also need to use {} brackets to ensure that the password is a least 8 characters long. Here is the code:

strongPassword = re.compile(r'''(
    (?=.*\d) 
    (?=.*[a-z])
    (?=.*[A-Z])
    [^\s]{8,}
    )''', re.VERBOSE) 

res = []                            
for potentialPassword in strongPassword.findall(text):
    res.append(potentialPassword)                        
    print(potentialPassword)
    

The easiest one is the last statement, which says accept all non-space containing strings that follow the lookaheads above and are at least 8 characters long. Each lookahead follows the same format.

(?=.*\d), which is equivalent to (?=.*[0-9]) says match any character that is a digit. Similarly, the other two regex components say match any character that is a lowercase alphabet and uppercase alphabet respectively.

We then just output our results.

You can find the full code here at Github.

Personally, I found this really difficult mainly because I didn’t know how to chain multiple lookaheads together. Simply putting them one after the other without the .* required the characters to be ordered (lowercase first etc.). Nesting them like a bad if..else statement provided similar results. I ultimately found the correct solution here on stackoverflow, through an implementation for jquery, although not without surfing for over 1.5 hours.

Moreover, the implementation in the official regex website was absolutely nightmarish. A long, single line that was overflowing onto the right side of their webpage. Turns out that their implementation doesn’t even work, for reasons I couldn’t comprehend.