Album making troubles
May 29th, 2023 by Me
This is a continuation of my previous article! Please read it below.
There are 145,170 files and 775 folders in my father's picture archive. It totals a massive 358 GB in size, according to Windows File Explorer. I have to make this completely automated.
I also just found out that the face_recognition library I was using has a default threshold percentage of 60%, meaning that if a person detected has a 60% chance of being the same as a person in the dictionary, it counts. By the time my 1.0 version finished running, it had identified 668 "unique" people. By the 668th person, the probability of correctly identifying a unique person is 6.38 * 10^-149. This is so comically low of a success rate that the number of unique atoms in the universe is about 10^80. You are more likely to pick the same atom out of all the atoms in the universe than for my program to run as intended. Wow! The false positive percentage is too high. What to do? Well, the solution could be to have a very small dictionary of photos, and make sure these photos are incredibly high quality.
Now, how do we make this dictionary? Tthe issue is not iterating through all ~145k images. The issue is the facial recognition and the false positive rate.
Album making troubles
May 29th, 2023 by Me
I recently obtained my father's collection of about 400 gigabytes of family photos from the last thirty-plus years. It's fun seeing people change over time and seeing baby photos. Some people passed away a long time ago, and some people had great things come to them which they had no idea. Anyway, this 400 gigabytes of images is really unwieldly. I want to know which photos contain photos of only me, for example. How should one even tackle this problem?
I started out by thinking: "Well, I know Python, and I know it has great support for machine learning libraries, so why don't I use use one?" I downloaded the "face_recognition" library and after a few hours of messing around with WSL2 for greater performance, stability, and support, I was soon able to recognize people from my photos, just as advertised. My original plan went like this:
- Parse through every image in the directory.
- Find every face in the photo.
- If a new person is detected, add them to the list of detected people and make for them a list of images where they appear. If this person has appeared before, add this image to their list of images.
Straightforward, right? It seemed that Google Photos's own method works in a similar way. The biggest advantage was creating a list of images because duplicating images would use at least 400 GB or more. When I ran the program the first time, I was able to express everyone who appeared in just 270 MB. That is a 99.9% decrease in storage usage compared to the original. Although it did end up running, there were certain issues with the approach.
Firstly, some key assumptions have to be made to have this program run as generic as possible:
- All that is given is an input directory and an output directory. The program cannot assume there are no subfolders and must account accordingly.
- Some photos may be corrupt, some items might not be images, or some images may not contain any faces at all. Any of these instances cannot break the program execution.
- Start with a completely empty dictionary of recognized people. Fill up the dictionary with new people. (This ended up being a big issue that I will go into later.)
To solve the first assumption, I wrote a recurseive depth-first-search algorithm that first scans a directory for photos, processes them, then recurses into the first folder. Repeat this process until no further directories exist (which is another assumption but is pretty obvious when you think about it. There cannot be an infinitely extending subfolder nightmare!) This had the huge added benefit for my purposes because the program automatically processes each folder alphabetically, and because each folder is at least vaguely organized by year, the photo processing was at least chronological. I also wanted a way to keep track of its progress, so I had the program print out whatever directory it recursed into or recursed out of. This is important because I could see when a folder was finished. Because everything was chronological, I could generally see how long the program was taking (and it sure did take a long time...).
With this 1.0 version of the program, I set it to run and left it. Photo processing is very fast, and because I didn't bother trying to implement CUDA, I just ran it on the CPU. Within an hour it was done with the first couple years. Unfortunately, it got extremely slow, and it ended up taking four days! What was worse was that it wasn't even using 100% of the CPU all of the time, and it wasn't the reads and writes that were slowing it down. What ended up being the culprit was the "simple" task of checking if a person has appeared before. This ended up being a HUGE flaw in the design, which I will explain more shortly. To speed it up, I decided to multithread the program by essentially run each check for each year folder. So 2001 got its own version of the program that run, and so did 2019 and every year inbetween. The reason why this would work is that the program doesn't know who these people are. It would just label "Person 1" then "Person 2" and so on for each person it detected. All I would have to do is go back and compare each year to each person that appeared. For example, "Person 1" in 2008 might actuallly be "Person 38" in 2005. It worked well and this multithreaded approach finished in about a day. Still slow, but still a huge improvement.
When the program finished after four days, I went through all 600 or so unique people it detected and started labeling them. It was then where I realized there was a HUGE problem that some readers may have spotted already. The probability of a false positive is extremely high! I assumed that each "unique" person would appear just once, but I don't appear at all! Every time the program finds a new person, it has to check if it exists in the database. This problem actually has to do with the slippery slope fallacy.
The slippery slope fallacy can be explained through US foreign policy from the '50s. They called it the Domino Effect, where if one country fell to Communism, their neighbors might do so as well. The problem with the fallacy is like this: Let's say there are 20 events that need to be satisfied to guarantee that a neighbor country falls to Communism. Even if each event has a probability of 95%, the probability the final event will happen is only 36%! I bring this up so people do not fall victim to this fallacy.
In my program, even if a completely unique person is checked against the dictionary and each face has a 99.5% chance of correctly identifying that these two faces don't match, by the 668th person (which is how many my program detected in total), the probability this person will stay unique has decayed to an astonishingly low 4% chance. This is pretty much unacceptable. I was thinking I could optimize the program by sorting the dictionary by frequency appeared, but now I can't even be sure of the frequency of people appearing!
After a nice long break of realizing all my work was pretty much useless, I noticed this program actually combined two key functions together. The first was the face detection, and the other was comparing. I could break up the program to see if a person detected is a valid face, manually approve it, and allow it to be sorted. This doesn't address the slippery slope fallacy problem. The approach I came up with was to instead make the dictionary myself. Manually select images of certain important people and run all 400 GB to see where they appear. It is better to have a small, focused dictionary because more people would only decay the accuracy exponentially. I will program this approach and return with an update.
If you have gotten this far, I greatly appreciate you reading this huge technical writeup. Let's get in touch!
The week of web dev
May 23rd, 2023 by Me
There are a lot of JavaScript frameworks out there, like React, Vue, Svelte, and many others. Determining the difference between Vanilla and a framework can boil down to a few key elements. For example, there are declarative and imperative items. Imperative is managing the DOM directly. Dealing with vanilla JS is like going with low level programming. Even though going low level has its advantages, building apps this way do not scale particularly well. Some of the reasons why it doesn't is because data is decoupled from the UI and must be manually updated. Another reason is that it is hard to tell if event listeners are attached to the plain html.
Based on the advice of Jeff from Fireship, there are three main questions that need to be answered when choosing a framework. Is the website highly interactive, is it search engine friendly, and is there dynamic content.
If the website is NOT highly interactive, using Hugo, 11ty, or Astro should be enough.
If it is highly interactive but does NOT need to be search engine friendly, just make a single page app using your favorite framework, like Alpine, Svelte, Vue, or React.
If you do need to be search engine friendly but the content does NOT change much, your app sounds like a Jamstack application, which can be satisfied using Next.js, Jamstack, or Gatsby.
Lastly, if you answered yes to all three, you most likely need full server-side rendering, through Next, Nuxt, or SvelteKit. He recommends you default with Next, Nuxt, or SvelteKit if you are not sure how your project is going to evolve.
There are a lot of ways to make a FullStack app, and it would be unfortunate if this much variety did not exist. I spent the past week with the challenge of creating one app per day. I made a Tetris clone, a Fidget Spinner simulation, a Double Pendulum simulation, and others. These apps have no database or SSR and run entirely in the web browser. They don't need it!
I am a big fan of minimalism, particularly when starting new projects or endeavors. I CAN throw a ton of money, time, or compute power at a problem or website, but all that matters in the end is if the users have a good experience with the app. When starting with JS web development, it makes sense to using document.getElementID and other low level interactions with the DOM just to get an idea of what's really going on under the hood, that way you can better appreciate and understand when you can introduce an @click into a button element for example. I have been using Bootstrap for my CSS because I like the premade elements. Understanding how CSS works is pretty complicated at first, and I like to learn by example and learning through low-level examples.
I think after this week, I can reasonably make basic interactive websites, but I have a long way to go, particularly for apps that are not only more than one page, but are also for other platforms like Windows, Mac, Linux, and especially IOS and Android. I have plans to make an app, I have the idea and concept and plan and even have the base core tech working, but when I started out, I didn't really understand what the DOM even is. I think it's nice to start small and build up, as I can reasonably say that I have leveled up a level of competence and hope that I can continue to level up with mobile and desktop apps!
New domain!
May 19th, 2023 by Me
I was successfully able to move this domain from GitHub to qayyumayaan.dev! I am extremely happy that I can definitively say I have my own corner of the internet now. I was pleasantly surprised at how easy it was to enable, just by going to domains.google.com and filling out a few items and I was off to the races. I highly recommend anyone interested in carving out their portion of the internet to do the same. I would be glad to give advice or more detailed instructions, particularly on hosting. Thank you!
Hello, world
May 16th, 2023 by Me
Even though the title of this blog is very corny, I'm glad I decided to sit down and make my own personal website from scratch. I had such a pain with templates and trying to take the easy way that I just decided it would be easier if I wrote everything myself. I settled on petite-vue and bootstrap because of how simple it is to use and how little overhead they provide (just a few KB)! These past two days I have had more commits to GitHub than ever before. I am very proud at how this came out. I learned a lot about HTML and CSS along the way. I want to turn this website into a template for others to use as well. Given how I set it up, it shouldn't be that hard. The biggest issue for me was just getting the project to run on GitHub. I did it the way I knew how to, with an index.html file.
Do let me know if there is any unexpected behavior on this site. And feel free to contact me via my email!