In early August, I got the chance to attend the Capital One Software Engineering Summit in Clarendon, Virginia. During my time at the summit, I participated in a financial-service themed hackathon, where my team developed Capitalize, a full-stack web application in Angular + Node that allows users to find full travel packages within a specified budget. (The link to the github repo for this project can be found here).

Implementation

Capitalize Design Overview

The frontend of Capitalize was written in Angular, while the backend was written in Node.js + Express. This allowed us to create a webservice on the backend that we could easily call from the frontend with a simple REST call.

The user would first enter in their budget, category of preference, number of travelers, and length of vacation. The frontend would then compile this information and send it to the backend through the rest call. The backend would then take these parameters and aggregate data from different sites (the Yelp API, the Google Flights API, and the Amadeus API) and return a JSON array to the frontend that contains the full vacation packages as JSON objects. The frontend would then render these in the view.

Design Choices and Takeaways

At this point in the web application, we could have created the entire app in Angular, since it would be able to aggregate data and call external APIs itself. However, I thought it would be better to abstract it into a frontend and a backend so that in the future, both could scale more efficiently and independently.

This application also was a good lesson in passing data between components in Angular. While sharing data between parent and child can be as simple as a viewChild, passing data between independent components required the use of services.

During the later part of my time at the Rice Efficient Computing Group, I developed the virtual filesystem to expose kernel components in an organized manner and to temporarily store files. Essentially, the virtual filesystem I wrote serves as the layer between the user interface and more concrete filesystems, such as a network filesystem or the physical filesystem on disk. (The link to the github repo for this project can be found here).

One of the most difficult (and simultaneously easiest) parts of this project was writing it in Rust. Rust has a super strict typesystem, so it made it difficult unifying the different types of files and directories across one single tree structure, and it necessitated the extensive use of pointers and type-casting. However, Rust’s memory safety guarantees allowed me to code pointer-heavy structures without fear of null-pointer errors or other memory-related bugs common to C-languages.

This project also led to my first Stack Overflow question. (I ended up sticking with the Vec<Box<Node>> idea so that more classes could implement the Node trait in the future).

Notes on Implementation

The VFS project led to the development of certain common unix commands for traversing filesystems, such as cd, ls, mkdir, rm, and pwd. By far the most difficult command to implement was cd, which changes directories. It biggest reason for this was because it involved creating the logic for relative/absolute paths. It was a thorough exercise in string parsing, recursion, tree traversal, and error handling. For example, here is the get function, which would return the node based on the wd (working directory) parameter:

    /// Gets the reference to the directory specified by the path given the current working directory 
    pub fn get(&self, wd: &StrongAnyDirRef) -> Result<FSNode, &'static str> {
        let current_path;
        { current_path = Path::new(wd.lock().get_path_as_string());}
        
        // Get the shortest path from self to working directory by first finding the canonical path of self then the relative path of that path to the 
        let shortest_path = match self.canonicalize(&current_path).relative(&current_path) {
            Some(dir) => dir, 
            None => {
                error!("cannot canonicalize path {}", current_path.path); 
                return Err("couldn't canonicalize path");
            }
        };

        let mut new_wd = Arc::clone(&wd);
        debug!("components {:?}", shortest_path.components());
        let mut counter: isize = -1;
        for component in shortest_path.components().iter() {
            counter += 1; 
            // Navigate to parent directory
            if component == ".." {
                let dir = match new_wd.lock().get_parent_dir() {
                    Ok(dir) => dir,
                    Err(err) => {
                        error!("failed to move up in path {}", current_path.path);
                        return Err(err)
                        }, 
                };
                new_wd = dir;
            }
            // Ignore if no directory is specified 
            else if component == "" {
                continue;
            }

            // Navigate to child directory
            else {
                // this checks the last item in the components to check if it's a file
                // if no matching file is found, advances to the next match block
                if counter as usize == shortest_path.components().len() - 1  && shortest_path.components()[0] != ".." { // FIX LATER
                    let children = new_wd.lock().list_children(); // fixes this so that it uses list_children so we don't preemptively create a bunch of TaskFile objects
                    for child_name in children.iter() {
                        if child_name == component {
                            match new_wd.lock().get_child(child_name.to_string(), false) {
                                Ok(child) => match child {
                                    FSNode::File(file) => return Ok(FSNode::File(Arc::clone(&file))),
                                    FSNode::Dir(dir) => {
                                        return Ok(FSNode::Dir(Arc::clone(&dir)));
                                    }
                                },
                                Err(err) => return Err(err),
                            };                       
                        }
                    }
                }
                               
                let dir = match new_wd.lock().get_child(component.clone().to_string(),  false) {
                    Ok(child) => match child {
                        FSNode::Dir(dir) => dir,
                        FSNode::File(_file) => return Err("shouldn't be a file here"),
                    }, 
                    Err(err) => return Err(err),
                };
                new_wd = dir;
            }

        }
        return Ok(FSNode::Dir(new_wd));
    }
}

This function wasn’t even a part of the public interface, but it was one of many helper functions that did a lot of the behind-the-scenes work for the virtual filesystem.

Takeaway

Over the 4+ month-long development process, I changed the public interface at least 10 times. Every time I changed it, something would break internally and would require hours of debugging to find the error.

The biggest lesson I learned from this project was to create an interface at the onset that is flexible enough to support future endeavours so that I can avoid days’ worth of debugging in the future.

Check out the live demo here. The git repo for this project can be found here.

This project was done for a MindSumo challenge, where the objective was to create a web application that would serve as an information kiosk/search engine for U.S. national parks and related destinations. I used the Python-Flask framework because I had previously used it in a hackathon before and I wanted to become more proficient with it.

One of the reasons this projects was so important to me was because it incorporated different ideas that I had used individually in previous projects, but not together simultaneously. This included the Google Maps API, Javascript, way too much CSS, and building an internally consistent website.

Notes on Implementation

One of the hardest things in this project was preventing code duplication. I refactored several times just so that I could add more features in the future without confusing myself. Another difficult component was using the somewhat fidgety National Park Service API. It would often return empty fields (such as empty image url fields) which would require some extra error handling so that the page could render on these events.

Takeaways

In retrospect, I could have used Angular (which I was using for a project at work at that time) for this NPS application. I like the modularization that Angular brings and its benefits, such as distinct components in a view that can be individually updated. With Flask, I found myself copying the navbar code across every webpage when I could have used the Angular router to only update the middle page content. I just realized custom HTML tags existed :/ However, Flask was very intuitive to use and was great for people like me without a lot of front-end experience.

The code for the terminal can be found here.

The terminal was my first major project in Rust as part of the Rice Efficient Computing Group. My objective was to create a unix-like terminal to allow users to run commands/applications in the operating system.

Starting Out

I had available three libraries (or “crates” in Rust):

1. library to capture keypresses
2. library to display text to a display buffer
3. library to run kernel tasks

Stages of Implementation

I began by creating the functionality to run commands in the kernel. Using the 3rd library, I wrote a function to validate an input against existing commands and then execute the preset command upon validation.

With this functionality in place, I moved to emulating terminal behavior. This involved recording user input, tracking/resetting prompts, mapping keypresses to certain actions, and echoing keypresses back to the terminal. It resulted in a very rudimentary terminal; however, the code was extremely messy and confusing. As a result, I added in the idea of standard input/output in the terminal for greater orthogonality.

The final major stage was to beef up the terminal. I implemented a few common unix commands, better error handling, and some other features you would find in an everyday terminal like command history.

Notes

The most difficult part in the early stages of the terminal was working with raw strings. Since at the end of the day, the display buffer only displayed the raw string that was passed to it, I had to meticulously index the display string to prevent out-of-bounds and overlap errors.

Another concern that arose in the later stages of development was the separation between external libraries and internal components and commands. Take the ls command, for example. It would make sense to put ls as a separate application in a different crate than the terminal, but that would also incur large overheads everytime the command was invoked (because the kernel has to start a new process and context switch). However, leaving the logic inside the terminal would bloat the code. For decisions like this, we would bias towards externalizing a lot of this logic for the sake of maintainability.

Takeaways

Real-time input handling is hard. For example, this was the code that redirected keypresses and ran built-in functions.

    loop {
        // Handle cursor blink
        if let Some(text_display) = terminal.window.get_displayable(&display_name){
            text_display.cursor_blink(&(terminal.window), FONT_COLOR, BACKGROUND_COLOR);
        }

        // Handles events from the print queue. The queue is "empty" is peek() returns None
        // If it is empty, it passes over this conditional
        if let Some(print_event) = terminal.print_consumer.peek() {
            match print_event.deref() {
                &Event::OutputEvent(ref s) => {
                    terminal.push_to_stdout(s.text.clone());

                    // Sets this bool to true so that on the next iteration the TextDisplay will refresh AFTER the 
                    // task_handler() function has cleaned up, which does its own printing to the console
                    terminal.refresh_display(&display_name);
                    terminal.correct_prompt_position = false;
                },
                _ => { },
            }
            print_event.mark_completed();
            // Goes to the next iteration of the loop after processing print event to ensure that printing is handled before keypresses
            continue;
        } 


        // Handles the cleanup of any application task that has finished running, including refreshing the display
        terminal.task_handler()?;
        if !terminal.correct_prompt_position {
            terminal.redisplay_prompt();
            terminal.correct_prompt_position = true;
        }
        
        // Looks at the input queue from the window manager
        // If it has unhandled items, it handles them with the match
        // If it is empty, it proceeds directly to the next loop iteration
        let event = match terminal.window.get_key_event() {
            Some(ev) => {
                ev
            },
            _ => { continue; }
        };

        match event {
            // Returns from the main loop so that the terminal object is dropped
            Event::ExitEvent => {
                trace!("exited terminal");
                window_manager::delete(terminal.window)?;
                return Ok(());
            }

            Event::ResizeEvent(ref _rev) => {
                terminal.refresh_display(&display_name); // application refreshes display after resize event is received
            }

            // Handles ordinary keypresses
            Event::InputEvent(ref input_event) => {
                terminal.handle_key_event(input_event.key_event, &display_name)?;
                if input_event.key_event.action == KeyAction::Pressed {
                    // only refreshes the display on keypresses to improve display performance 
                    terminal.refresh_display(&display_name);
                }
            }
            _ => { }
        }

I learned to be careful about what functions I was calling in this loop to prevent “keypress lag”, because each iteration of this loop would track a single keypress. It was a good exercise in writing efficient code!

The git repo for this project can be found here.

When my family and I were driving to Colorado back when I was ten, I remember my mom asking where the nearest gas station was and me thinking that it would be really convenient if Google Maps allowed me to search for gas stations along our current route. That inspired this project, though Google Maps later implemented this feature anyways.

The purpose of this application is to find gas stations, restaurants, etc. along a route within a specified distance from the route. It uses the Python-Flask web framework to serve the application.

Challenges

One of the challenges I faced was that the Google Maps API put a query per second limit on calls to its service. I originally tried to query the Maps API for destinations at small, preset increments along the route, but I ran into the aforementioned query limit. As a workaround, I queried the Google Maps API at 20 equally spaced intervals around the route with a radius backwards-calculated from the user-specified distance. I then calculated the shortest distance from any location from the route to ensure it was less than the user-specified distance.

In this process, I learned about the haversine distance, which is the shortest distance between two objects on a globe. The javascript function I used is shown below:

      function haversine(lat1, lon1, lat2, lon2) {
        var R = 6371; // km
        var dLat = toRad(lat2-lat1);
        var dLon = toRad(lon2-lon1);
        var lat1 = toRad(lat1);
        var lat2 = toRad(lat2);

        var a = Math.sin(dLat/2) * Math.sin(dLat/2) +
          Math.sin(dLon/2) * Math.sin(dLon/2) * Math.cos(lat1) * Math.cos(lat2);
        var c = 2 * Math.atan2(Math.sqrt(a), Math.sqrt(1-a));
        var d = R * c;
        return d;
      }

Takeaways

This project was a solid exercise in making workarounds to external factors that were out of my control (or more precisely, that I wasn’t willing to pay for). It was also my first introduction to Javascript.