The Google Graveyard

Cemetery

Google is a much beloved tech company. Many people are familiar with their products such as Chrome, the Google search engine, and the Android operating system. However, behind these well-known products lies a graveyard of failures and tossed out products. Among the biggest of those failures was Google+ – the short-lived attempt by Google to enter into the social media sphere. However, countless other products have enter’s Google’s Graveyard. Google Hangouts, a once-popular chat application is slated to be shut down in November of this year, and Google Chrome Apps was killed off earlier this year. For a full list, you can visit the Killed by Google website.

Clearly, businesses must determine what products are profitable and which ones are not. Those found to be unprofitable will be cut to free resources for more profitable products. However, this becomes more difficult when Google has such a huge influence in the technology sphere. For example, Google is responsible for Angular, Go, Dart, and Firebase. Each of these tools is used by developers around the world. Should they become obsolete by Google, they may find their way into the Google Graveyard as well.

Even worse, Google has a history of making significant changes to their development tools – changes which often break everything. For example, the change from AngularJS to Angular made the former absolute. Likewise, changes to the Firebase API which migrated from Promises to Subscriptions ruined any previous code.

My concern, as a developer, is that Google’s track record for tossing out old ideas may one day leave programmers without the tools they need. One day, Google may decide to toss out Angular for a new, fancier framework. Or, maybe they discontinue their Firebase service and leave scores of applications without a home. As such, I am now slower to adopt Google technologies, because I’m tired of watching useful tools make it into the Google Graveyard.

New Language Shortcomings

During the last 20 years, a huge number of new programming languages have been developed. In general, new languages have evolved to address the perceived shortcomings of older languages. Unfortunately, these languages may not always live up to the hype. Furthermore, new language ‘features’ may actually end up creating more complicated and unreadable code. I have previously written about the importance of simple code as well as the trap of “time saving” frameworks. However, not all new languages follow these ideals.

Recently, I have started learning Kotlin. Kotlin is a JVM language which is increasingly used for Android development. Thanks to a variety of improvements, Kotlin code can do more work with less code. However, this isn’t always a good thing. For example, Kotlin generates a significant amount of code behind the scenes. While this does reduce the burden to developers, it does not guarantee that the desired code will actually perform the necessary function. In fact, I have often found that the most difficult bugs to find are those generated by code under the hood. For example, I remember using Hibernate in the 2000’s to connect Java to the database. In 99% of my use cases, Hibernate performed incredibly well and saved significant time. However, the 1% of cases where the SQL generated by Hibernate didn’t perform properly required enough research and work to remove any possible time savings to the developer.

Another example with Kotlin, and a large number of newer languages, is the lack of requirements to actually define variables and return types for functions. While this may save time, it means that future developers will be forced to read the code more closely to determine its meaning. Likewise, the growing use of lambda functions – while incredibly useful in a broad assortment of situations – is beginning to create code that is nothing but extended sequences of lambda functions with no obvious intent. In both instances, I prefer verbose code that can be easily understood by future developers with far less chance for misunderstandings.

While many of these newer languages have some amazing features, few have actually captured much market share. Maybe it’s because of the glut of languages on the market now, or maybe it’s because developers have found these ‘improved’ languages actually cause more problems than they solve. For me, I will continue to write code with the hope that junior developers will not be wowed by my wizardry, but rather that they will be able to understand my code without any ambiguity.

Mature Frameworks

As a seasoned software developer, one of the things I most value is mature frameworks. By mature, I don’t necessarily mean old. Rather, I mean a framework that will still be useful tomorrow. Furthermore, that framework should have documentation that will help me today and still be relevant in the future.

Languages like C, C++, and Java provide these frameworks. While Java has had a number of revisions since it was created, code from the 20 years ago will run just as well as modern code. While it may not take advantage of newer features, it is no less viable. Furthermore, books on Java development from 15 years ago can still provide new developers with instruction on programming. In fact, the only part of Java that has really been significantly obsoleted is the windowing frameworks. Likewise, C and C++ code from the 90’s will still run well today, with some minor tweaking.

On the flip side of the spectrum lie the various Javascript frameworks. While Javascript has been around since the beginning of the internet, it seems that frameworks are constantly evolving. Consider the Angular framework. I recently was asked to update an application written in Angular just a few years ago. However, I quickly learned that all of the APIs were now obsolete. For example, the HTTP client had been obsoleted and other libraries wanted to use Observables instead of Promises. To make matters worse, the entire Firebase API was virtually scrapped in favor of an entirely new API.

What makes these changes even more difficult for developers is that the documentation is often poor or non-existent. For example, I attempted to run the newest Firebase API tutorial from Google only to learn that none of the code would compile with the newest libraries. I had to spend an hour looking for a more recent code sample. Try to find a current book on the Angular framework and you will quickly see the challenge.

Unfortunately, this seems to be the status par for Javascript frameworks. Each one has numerous changes and little documentation or training materials for developers. Yet, we continue to use these frameworks. Why? Because Javascript is the defect standard for web development. You have no other choice! If you did, you would certainly use a mature framework that would still compile tomorrow!

Custom Software – A Case Study

Custom Software

I recently contracted the services of a company to help me become a GSA contractor. While the company is working on submitting my paperwork, they have encouraged me to start reaching out to potential government clients. To help me find those clients, the company offered me thirty days of access to their website that shows all federal and state opportunities as well as providing search functionality and email notifications. However, after the thirty day trial, the price of accessing their site is $200 / month or $2,000 / year.

The Problem

While the company strongly encouraged me to purchase their services, I am always skeptical of paying for things I don’t need. As a business owner, wasted money comes directly out of my pocket. So, instead of paying for their services, I decided to examine alternatives. First, I found that much of the functionality was already available on the governments System for Awards Management (SAM). Second, I found that registered users of SAM can request an API key to develop their own software.

Developing a Custom Solution

Given that all I wanted was a simple app to display matching opportunities, I requested an API key and started development. To begin, I had a junior developer create an Angular web app. In order to access the SAM API, I created simple Node-based REST service. Next, I updated the Angular app to function as a PWA so that I can install it on my phone.

Outcome

Now, after less than 4 hours of development, I have an app on my cell phone to display opportunities matching my criteria. Or, I can access the site from my computer and go directly to SAM if I want more information. While there are many upgrades I could make in the future, the cost of developing my own custom software was substantially less than paying a third-party to use their service and delivered exactly what I needed.

Advice to Businesses

Today, a significant number of businesses offer Software-as-a-Service. While this model is great for the software provider, it may be less optimal for the consumer. Over time, the total cost of SaaS continues to rise for the consumer while the benefit remains largely the same. However, custom software allows for ownership of the application without a growing price tag. Furthermore custom software can address issues unique to the customer which may not be addressed by a Commercial Off-the-Shelf system.

In order to make the best decision for your business, consider the monthly cost of the application over a several year period. Then, consider the cost of lost productivity due to missing functionality. Once those costs are totaled, find out the cost of developing custom software to meet what you actually need. If the cost of custom development is less than the commercial solution, consider creating your own application.

Apple vs Android – A Developer’s Perspective

While most applications are developed for both iPhone and Android systems, new developers are faced with the choice of which platform to learn. While both Android and iPhone systems offer excellent apps as well as a variety of sizes, they differ considerably from a developer perspective.

Android Development

For the novice, Android development is probably the easier entry point. For starters, low end Android phones are cheaper to purchase than iPhones. But more importantly, Android developers can use Windows, Linux, or Mac machines for development. So, if you have a computer and an Android phone, you can get started right away.

The language used on Android phones is Java or Kotlin. While Kotlin is the newer language, more resources on Java development are available to get started. Furthermore, once you learn Java, you will find other development opportunities open up to you – such as backend services using frameworks such as Spring Boot.

Once you have learned how to program Android phones, you will find that other devices use Android as well. This includes Virtual Reality hardware such as Oculus, Augmented Reality glasses from vendors like Vuzix, and smart watches.

Publishing to Google is relatively simple too. Once you pay a one-time fee, you are a licensed developer and can create and deploy applications to the Google Play store. While there is some over sight from Google, it is less burdensome than Apple’s requirements.

iPhone Development

iPhone development is a little more complicated. For starters, you will need a Mac machine as the tools for iPhone development do not run under Windows or Linux. Furthermore, both Apple computers and iPhones tend to be more expensive for a small development setup.

While Android’s Java language is used everywhere, the iPhone’s Swift language is far more limited. In fact, Swift isn’t used outside of the Apple ecosystem. So, if you chose to develop other services to integrate with your phone, you will need to learn an additional language.

Unlike Android, few devices run iOS. Thus, your skills on iPhone development will not translate to the ability to program other devices aside from the Apple Watch.

Finally, Apple’s App Store is far more expensive and burdensome than the Google Play Store. For starters, Apple requires developers to pay an annual license fee – which is more expensive than Google’s one-time license cost. Furthermore, the Apple Store is much more strict with requirements for apps and provides significantly more oversight on the app market.

Conclusion

While I think both the Apple and Android phones are excellent, I personally find the Android developer experience to be more positive. This is particularly true for the indie developer or individual looking to learn mobile development.

What is Computer Vision?

Computer Vision is a rapidly growing technology field that most people know little about. While Computer Vision has been around since the 1960s, it’s growth really exploded with the creation of the OpenCV library. This library provides the tools necessary for software engineers to create Computer Vision applications.

But what is Computer Vision? Computer Vision is a mix of hardware and software tools that are used to identify objects from a photo or camera input. One of the more well-known applications of Computer Vision is in self-driving cars. In a self-driving car, numerous cameras collect video inputs. The video streams are then examined to find objects such as road signs, people, stop lights, lines on the road, and other data that would be essential for safe driving.

However, this technology isn’t just available in self-driving cars. A vehicle I rented a few months ago was able to read speed limit signs as I passed by and display that information on the dash. Additionally, if I failed to signal a lane change, the car would beep when I got close to the line.

Another common place to find Computer Vision is in factory automation. In this setting, specialized programs may monitor products for defects, check the status of machinery for leaks or other problematic conditions, or monitor the actions of people to ensure safe machine operation. With these tools, companies can make better products more safely.

Computer Vision and Artificial Intelligence are also becoming more popular for medical applications. Images of MRI or X-Ray scans can be processed using Computer Vision and AI tools to identify cancerous tumors or other problematic health issues.

From a less practical view, Computer Vision tools are also used to modify videos of user content. This may include things such as adding a hat or making a funny face. Or, it may be used to identify faces in an image for tagging.

Ultimately, Computer Vision technologies are being found in more and more places each day and, when coupled with AI, will ultimately result in a far more technologically advanced world.

Productivity Gains through Aliases & Scripts

For users of Mac or Linux-based machines, aliases and scripts can create some of the most valuable tools for increased productivity. Even if you run a Windows machine, there is a strong possibility that some machines you interact with – such as AWS – utilize a Linux-based framework.

So, what are aliases and scripts? Scripts are files that contain a sequence of instructions needed to perform a complex procedure. I often create scripts to deploy software applications to development server or to execute complex software builds. Aliases are much shorter, single line commands that are typically placed in a system startup file such as the .bash_profile file on MacOS.

Below are some of the aliases I use. Since Mac doesn’t have an hd command (like Linux), I have aliased it to call hexdump -C. Additionally, Mac has no command for rot13 – a very old command to perform a Caesar cipher which I have aliased to use tr.

Since I spend a lot of time on the command prompt, I have crated a variety of aliases to shortcut directory navigation including a variety of up command to move me up the directory hierarchy (particularly useful in a large build structure) and a command to take me to the root folder of a git project.

Finally, I have a command to show me the last file created or downloaded. This can be particularly useful, for example, to view the last created file I can simply execute cat `lastfile`.

alias hd='hexdump -C'
alias df='df -h'
alias rot13="tr 'A-Za-z' 'N-ZA-Mn-za-m'"
alias up='cd ..'
alias up2='cd ../..'
alias up3='cd ../../..'
alias up4='cd ../../../..'
alias up5='cd ../../../../..'
alias up6='cd ../../../../../..'
alias root='cd `git rev-parse --show-toplevel`'
alias lastfile="ls -t | head -1"

One common script I use is bigdir. This script will show me the size of all folders in my current directory. This can help me locate folders taking up significant space on my computer.
#!/bin/bash

SAVEIFS=$IFS
IFS=$(echo -en "\n\b")

for file in `ls`
do
        if [ -d "$file" ]
        then
                du -hs "$file" 2> /dev/null
        fi
done
IFS=$SAVEIFS

Another script I use helps me find a text string within the files of a folder.
#!/bin/bash

SAVEIFS=$IFS
IFS=$(echo -en "\n\b")

if [ $# -ne 1 ]
then
        echo Call is: `basename $0` string
else
        for file in `find . -type f | cut -c3-`
        do
                count=`cat "$file" | grep -i $1 | wc -l`
                if [ $count -gt 0 ]
                then
                        echo "******"$file"******"
                        cat "$file" | grep -i $1
                fi
        done
fi
IFS=$SAVEIFS

These are just a few examples of ways to use scripts and aliases to improve your productivity. Do you have a favorite script or alias? Share it below!

Basics of Artificial Intelligence – IX

After an artificial intelligence algorithm is selected and trained, the final step is to test the results. When we started, we split the data into two groups – training data and testing data. Now that we’re done training the algorithm with the training data, we can use the test data for testing. During the testing phase, the algorithm will predict the output and compare to the actual answer. In most datasets, some data will be incorrectly predicted.

The confusion matrix allows us to get a better understanding of the incorrectly predicted data by showing what was predicted vs the actual value. In the below matrix, I trained a neural network to determine the mood of an individual based on characteristics of their voice. The Y axis shows the actual mood of the speaker and the X axis shows the predicted value. From my matrix, I can see that my model does a reasonable job predicting fear, surprise, calm, angry, and happy but performs more poorly for normal and sad. Since my matrix is normalized, the numbers indicate percentages. For example, 87 percent of afraid speakers were correctly identified.

Creating the above confusion matrix is simple with Scikit-Learn. Start by selecting the best model and then predict the output using that classifier. For my code below, I show both the normalized and standard confusion matrix using the plot_confusion_matrix function.

# PICK BEST PREDICTION MODEL
classifier = mlp

# predict value
pred = classifier.predict(X_test)

# plot non-normalized confusion matrix
titles_options = [("Confusion matrix, without normalization", None),
                  ("Normalized confusion matrix", 'true')]
for title, normalize in titles_options:
    disp = plot_confusion_matrix(classifier, X_test, y_test,
                                 display_labels=data[predictionField].unique(),
                                 cmap=plt.cm.Blues,
                                 normalize=normalize)
    disp.ax_.set_title(title)

plt.show()

With the above matrix, I can now go back to the beginning and make changes as necessary. For this matrix, I may collect more samples for the categories that were incorrectly predicted. Or, I may try different settings for my neural network. This process continues – collecting data, tuning parameters, and testing – until the solution meets the requirements for the project.

Conclusion

If you’ve been following along in this series, you should now have a basic understand of artificial intelligence. Additionally, you should be able to create a neural network for a dataset using Scikit-Learn and Jupyter Notebook. All that remains is to find some data and create your own models. One place to start is data.gov – a US government site with a variety of data sources. Have fun!

Basics of Artificial Intelligence – VIII

Neural Networks are an incredibly useful method for teaching computers how to recognize complex relationships in data. However, in order to get them working properly, you need to know a little more about how they work and how to tune them. This week, we’ll be looking at the two key settings for Neural Networks in scikit-learn.

What Is a Neural Network?

But before we go into those settings, it’s useful to understand what a neural network is. Neural networks are an attempt at modeling computer intelligence on how the human brain works. In our brains, neurons receive electrical impulses from other neurons and, optionally, transmit impulses to other neurons. From there, the process continues with those neurons again deciding how to act on the signal from the previous neuron. Our brains have an estimated 100 billion neurons, all connected to the network to receive and process data.

In the computer, this same idea is replicated with the Neural Network. The inputs values for the network form the first layer of neurons in the artificial brain. From there, one or more hidden layers are created connecting the inputs from the previous stage. Finally, one or more output neurons provide the user with the answer from the digital brain. Of course, this assumes the network has been trained to identify the data.

So, for the developer, the first step to creating the neural network is to determine the number of layers for the network and the number of neurons in each layer. Next, the developer will select from a group of ‘activation functions’ that will define when the neuron fires. The available options are the logistic sigmoid function (logistic), the hyperbolic tan function (tahn) and the rectified linear unit function (relu). Various other parameters can also be set to further tune the network.

Back to the Code

# Create a Neural Network (AKA Multilayer Perceptron or MLP)
# In this example, we will create 3 hidden layers
# The first layer has 512 neurons, the second 128 and the third 16
# Use the rectified linear unit function for activation (RELU)
# For training, iterate no more than 5000000 times
mlp = MLPClassifier( 
    hidden_layer_sizes=(512,128,16)
    activation='relu',
    max_iter=5000000
)

You can see in the above code that we are going to try with 3 layers in the network. This is simply a guess, and we will want to repeatedly attempt different network configurations until we come upon a model that performs to the required specifications.

# Train the neural network
mlp.fit(X_train,y_train)

# Get metrics 
train_metric = mlp.score(X_train, y_train)
test_metric = mlp.score(X_test, y_test)

pred = mlp.predict(X_test)
recall_metric = recall_score(y_test, pred)
precision_metric = precision_score(y_test, pred)

With the above code, we can retrieve scores indicating how well the model did. With a perfect network, all values would be 1 – meaning they were 100% accurate. However, this is rarely the case with actual data. So, you will need to determine what level of accuracy is required. For some networks, 80% may be the limit.

Armed with this information, you should now be able to repeatedly train your network until you have the desired output. With a large dataset, and a large number of configurations, that may take a substantial amount of time. In fact, the training and testing part of AI development is by far the most time consuming.

What’s Next?

Next week, we will look at the final part of developing an AI solution – the Confusion Matrix. This chart will give us a better understanding of how our network is performing than the simple metrics we calculated above.

Basics of Artificial Intelligence – VII

Last week, we used Python libraries to import the data, set the input and out values for the computer to learn, and split the data into groups. Next, we will actually train the computer to learn the relationships. For this, we can use a variety of different tools. While each one has its pros and cons, the novice can simply run each one and determine which one provides the best results. In addition, we will print the results for analysis.

Logistic Regression

# train the model
logreg = LogisticRegression()
logreg.fit(X_train, y_train)

# print accuracy
train_metric = logreg.score(X_train, y_train)
test_metric = logreg.score(X_test, y_test)
print('Accuracy of Logistic regression classifier on training set: {:.2f}'.format(train_metric))
print('Accuracy of Logistic regression classifier on test set: {:.2f}'.format(test_metric))

# print recall
pred = logreg.predict(X_test)
recall_metric = recall_score(y_test, pred, average=recall_average)
precision_metric = precision_score(y_test, pred, average=recall_average)
print('Recall of Logistic regression classifier on test set: {:.2f}'.format(recall_metric))
print('Precision of Logistic regression classifier on test set: {:.2f}'.format(precision_metric))

Decision Tree Classifier

# train the model
clf = DecisionTreeClassifier().fit(X_train, y_train)

# print overall accuracy
train_metric = clf.score(X_train, y_train)
test_metric = clf.score(X_test, y_test)
print('Accuracy of Decision Tree classifier on training set: {:.2f}'.format(test_metric))
print('Accuracy of Decision Tree classifier on test set: {:.2f}'.format(train_metric))

# print recall/precision
pred = clf.predict(X_test)
recall_metric = recall_score(y_test, pred, average=recall_average)
precision_metric = precision_score(y_test, pred, average=recall_average)
print('Recall of Decision Tree classifier on test set: {:.2f}'.format(recall_metric))
print('Precision of Decision Tree classifier on test set: {:.2f}'.format(precision_metric))

Linear Discriminant Analysis

# train the model
lda = LinearDiscriminantAnalysis()
lda.fit(X_train, y_train)

# print overall accuracy
train_metric = lda.score(X_train, y_train)
test_metric = lda.score(X_test, y_test)
print('Accuracy of LDA classifier on training set: {:.2f}'.format(train_metric))
print('Accuracy of LDA classifier on test set: {:.2f}'.format(test_metric))

# print recal
pred = lda.predict(X_test)
recall_metric = recall_score(y_test, pred, average=recall_average)
precision_metric = precision_score(y_test, pred, average=recall_average)
print('Recall of LDA classifier on test set: {:.2f}'.format(recall_metric))
print('Precision of LDA classifier on test set: {:.2f}'.format(precision_metric))

Neural Network

# activation - ‘identity’, ‘logistic’, ‘tanh’, ‘relu’

mlp = MLPClassifier( 
    hidden_layer_sizes=(512,768,1024,512,128,16)
    activation='relu',
    learning_rate='adaptive',
    max_iter=5000000
)

mlp.fit(X_train,y_train)

# print overall accuracy
train_metric = mlp.score(X_train, y_train)
test_metric = mlp.score(X_test, y_test)
print('Accuracy of Neural Network classifier on training set: {:.2f}'.format(train_metric))
print('Accuracy of Neural Network classifier on test set: {:.2f}'.format(test_metric))

# print recall
pred = mlp.predict(X_test)
recall_metric = recall_score(y_test, pred, average=recall_average)
precision_metric = precision_score(y_test, pred, average=recall_average)
print('Recall of Neural Network classifier on test set: {:.2f}'.format(recall_metric))
print('Precision of Neural Network classifier on test set: {:.2f}'.format(precision_metric))

What We did

You will notice that much of the code above is very similar. This is part of what makes Scikit-Learn such an amazing framework – it’s relatively easy to change between Artificial Intelligence algorithms. In addition to the above algorithms, you can also use Support Vector Machines, Naive Bayes, K-Nearest Neighbor, and many more.

Once you’ve run the training, the scores show how each algorithm performed after it was trained. On any given data set, a different algorithm may work better. This is another benefit to Scikit-Learn – the easy access to a variety of models allows for experimentation to find the best model.

What Next?

While much of underlying math for these algorithms is well outside the scope of knowledge for most, it is useful to understand how Neural Networks operate. They are one of the more interesting implements of AI, and can be tuned to work with lots of data. However, that tuning requires some knowledge of what a Neural Network is and how it works. That’s what we’ll look at next week.