Tag: engineering

Posted on

Languages vs Algorithms

Programming

I was recently discussing a project with a potential client. His project was using a language I haven’t used heavily in quite some time. He was disappointed because he was struggling to find developers to work on his project. I indicated that I would not have trouble getting back up to speed on the language and being able to move his project forward.

This really highlighted in my mind what really matters in development – it’s not languages, it’s algorithms and software design. All programming languages are, at their core, pretty similar. For example, all languages include:

  1. Syntax for creating variables,
  2. Selectively executing a block of code based on one or more conditions
  3. Repeating a portion of code multiple times
  4. Encapsulating code into functions

These basic structures are responsible for all the software you will ever use. Whether it’s a complex operating system like Windows or MacOS, a cell phone application on iOS or Android, or an application on the cloud, they all rely on the basic elements above.

Once you understand the basic syntax of a language, you move into the more important stuff – algorithms. Algorithms are a set of instructions to perform a given task. Regardless of the language, the algorithm will be largely the same. Next up on the hierarchy is software design. Knowing syntax and algorithms is great, but to really develop quality software you need to be able to design the interactions between components. None of these things vary substantially between languages. Languages are merely the syntactic rules you must follow to define your algorithms and implement your design.

I think people need to worry less about what language a developer uses and look for higher-order skills including algorithms and design. That’s what really defines the skill of a programmer!

Posted on

MATLAB

Code

Running my own software company, I run into people trying to solve all kinds of different problems with software. As such, I end up using a variety of different technologies, languages, platforms, and frameworks. Recently, a client project required me to download MATLAB in order to experiment with several existing toolkits needed for the R&D phase of the project. The first thing I noticed was the price tag. Today, the overwhelming majority of development environments are free. However, MATLAB carries a hefty price tag of $800/year for commercial customers. After installing the application, I found out that the plugins (called toolboxes) averaged an additional $500/year. All told, this development environment was going to cost me around $3000/year after all necessary plugins were purchased.

Is It Worth It?

Seeing that MATLAB is so expensive, and so highly regarded by many in the research and academic community, I assumed that the MATLAB user experience was going to be epic! How wrong I was. The code I was examining contained dozens of files across numerous folders. Of course, this isn’t abnormal for any large project and I expect any decent IDE to be able to handle it. Any IDE except MATLAB. Need to find where a function is defined? Don’t expect any tools in MATLAB to accomplish that! Of course, you can do a find in files for all references of a string, but that’s not particularly helpful when a function is called dozens of times or has a short name. How about displaying a call hierarchy so that you can see where functions are being called? Nope – that’s not possible either. Ever aspect of the IDE proved to be well below what was commonly available in other IDEs.

If you’re not intersected in the user experience, I guess MATLAB is great. Apparently, its ability to perform all kinds of mathematical wizardry is beyond compare. It’s great for solving linear algebra problems and other advanced math tasks. Unfortunately, I wasn’t interested in that.

Open Source Code

I managed to find several open source projects and toolboxes for MATLAB. I was interested in understanding how these particular toolboxes work, so I started examining the code. MATLAB, like so many other languages, is a weakly-typed language. While weakly-typed languages do offer many benefits, readability is rarely one of them. MATLAB also does not require variables to be defined before use. This can be very difficult to follow particularly when the bulk of the code I saw contained neither useful variable names nor adequate comments to understand what’s going on. Very typical of people without a development background.

Conclusion

I was very disappointed with everything I saw in the MATLAB community. I assume most of the users are mathematicians and have no idea the kinds of features that are available in modern environments, so they have no idea what they’re missing. Likewise, I assume they’re used to digging through poorly documented code with variable names like g, f, c, and y. For me as a developer, this is neither acceptable nor maintainable long-term. For number crunching, I much prefer R. It may not have the most advanced IDE, but it doesn’t set me back anything either!

Posted on

Multilingual Programmers

One of the biggest difficulties in the computer programming world today is the sheer number of languages out there to use for application development. Which language should you learn? The reality is, that’s the wrong question. I recently spoke with a developer who indicated that he only knew C# – he was not comfortable with Python or any other language. For the professional developer, this it not an acceptable answer. In today’s world of programming, you can no longer get by knowing only a single language. For example, while the bulk of the code in a Java web application will be in Java, the front-end will require JavaScript and HTML. Build scripts may be written in Groovy or XML, and Shell scripts may be required for build automation. This is not unique to Java applications – just about any application will require some knowledge of other languages.

How do you pick which languages to learn? At a minimum, every professional engineer should know at least one compiled language such as Java, C, C++, or C# in addition to JavaScript, Python, and SQL. Additionally, he or she should be comfortable with another language such as Ruby, PHP, Go or Rust.

If you’re an application developer, do you only know one language? If so, I would strongly urge you to learn additional languages. Mastery of every language isn’t necessary, but any decent developer should feel capable of writing code in several languages.

Posted on

How I Got Here

During the last decade, I have been asked countless times how I would recommend someone get into the programming world without going to college. Many people don’t want to spend the time or money getting a computer science degree, and since I did it people generally assume I can tell them the shortcut to achieve the same in their life. Unfortunately, few people really realize the amount of time and effort it took for me to get where I am or the difficulties I’ve encountered by not having a degree in a field where a degree is expected.

I didn’t start working with computers until around 1997. I was in the US Army at the time, and worked exclusively with Unix machines. Unlike the Windows world, the Unix world has always had tools for developers. The machines I worked on had C, C++, Perl, Tcl/TK, Fortran, Bourne Shell, and numerous other programming tools. During the course of my work, I often used scripts written by others, and soon learned to modify them to do what I wanted them to do. I purchased programming books from Amazon and decided to learn more. At home, I setup an old computer to run Linux so I could have a similar development environment to the machines at work. As time went on, I decided to take courses through the National Cryptologic School on Perl, Unix Administration, C Programming, and other tech subjects. After becoming pretty comfortable with C, I decided to further my education through a correspondence school (the now defunct National Radio Institute) where I earned a diploma in Visual Basic Programming (as I was already pretty comfortable in C, I figured it best to branch out and learn a new language). A few years later, I took a few courses from the University of Maryland in C++ programming. Throughout this time, I spent countless hours at home learning everything I could about programming. I wrote programs to do all kinds of things – from GUI development to command line scripts. This was a difficult time in my life – working a full time job while spending all my free time learning to write computer software took a heavy toll on my marriage. But it would all be worth it when I got out of the Army and found a job in the software realm.

Then, in 2001, it was finally time to step out and find a programming job. Of course, I had no real experience as a programmer. I had written some code and scripts in the Army, but hardly anything that would really be considered production code. None-the-less, I managed to find a local company that needed an entry level programmer. And, thanks to a friend who knew the owner, I was offered a position. In my hubris, I assumed I knew everything at that point in my career. It would take years to fill the gaps in my knowledge to become a good programmer. With project after project under my belt, I would finally become a respected developer around a decade later.

What has my lack of degree caused me? During my entire career, I have found countless companies that wouldn’t even talk to me because I didn’t have a degree – they were unwavering on their requirements for a BS in Comp Sci even though I had been working in the field for years. At various companies I worked at, it became obvious that I would never be promoted simply because I didn’t have a piece of paper. Even though I could code better than my peers, my lack of degree held me back.

At no point has my path been easy. It’s involved an incredible amount of work and sacrifice. And if you’re thinking of teaching yourself to program and find a job, I wish you the best of luck as you are about to find out that it takes far more than watching a few online videos and making a webpage.

If you do choose this path, how can you make it successful? Passion – you must have an unwavering passion to write code. You need to spend every waking hour writing code, reading books, watching videos, and doing everything you can to become a good programmer. Expect to put in substantial time and effort. Expect to struggle finding your first job. Expect difficulty advancing in your career. Don’t think for a minute that it’ll be easy – I can promise you, it won’t.

Posted on

Picking a Server Platform

Many small businesses want websites or mobile applications that require server-side functionality. Often, this functionality includes a database to store user information. What options exist for a user? What are the pros and cons for each one? I will examine three different frameworks – Node and SQLite3, PHP and MySQL, and Tomcat. These represent just a small number of options available to a business – from small scale applications to enterprise solutions.

For small applications, I like Node and SQLite. Node is a simple platform to run server-side programming. Since it requires virtually no infrastructure, Node services can be installed and deployed in minutes.  Likewise, SQLite requires no installation. SQLite databases are a simple file that can be easily backed up or restored by copying the database file. While this framework is great for small applications, enterprise applications would benefit from more robust environments. Node and SQLite can work really well for small internal applications or to implement a small number of services using a small database.

Next up, PHP and MySQL. This combo is widely deployed on a variety of platforms. In fact, that’s one of the reasons I like it. Typically, service providers like GoDaddy have support for PHP and MySQL out of the box, so applications and services can be deployed without much effort.  PHP/MySQL is also more robust than Node/SQLite. On the negative side, PHP has a variety of versions that are substantially different and PHP code can easily become unmanageable if developers aren’t careful.  I like this solution for smaller customers needing a small number of services on an existing PHP server.

Finally, there’s Tomcat. This option is an excellent Java-based server bringing all the advantages of the Java programming language into a robust server environment. Tomcat can integrate with any database, but MySQL is a common solution. Tomcat is an excellent option for Java web applications or services, but suffers from one big problem – it’s the most complicated option to setup.  This option is best when a large number of users or a large database must be supported. This is the option I like to recommend for enterprise customers.

Numerous other databases and server platforms exist. Microsoft’s .NET platform can work great for customers who prefer Microsoft products and Ruby may be a desirable option for some customers too. As with all technology choices, server platforms must be selected based on customer requirements. Small customers appreciate rapid, low cost development while larger customers will want more robust solutions while being less price sensitive.

Posted on

Software as an Iterative Process

The world can broadly be divided into the physical world and the digital world.

In the physical world, products are manufactured and deployed. Those products never again see the manufacturer. If there is a problem, a new process may be implemented to solve the problem; but customers with the existing product will not likely see the benefit of the new process.

In the digital world, products are created and deployed just like in the physical world. However, everything that follows is different. When software problems are identified, patches are created or new versions are deployed. When the customer accesses the patches or upgrades their software, they see the benefit of the new changes.

This difference allows for a vastly different approach to creating and deploying products in the digital world. Unlike physical products, software can benefit from an iterative process. Software can be modified today, tested tomorrow, and deployed the following day. What if it doesn’t work? The changes can be rolled back or a new patch can be deployed. Unlike a physical product, a digital product is never complete.

This huge paradigm difference means that software companies can be far more nimble than manufacturing companies. Since changes can be made at any time, software companies enjoy far less risk than manufacturing companies.

As a developer of custom software, I often work with customers that are uncomfortable with this process. These customers want to wait until all functionality is present or until everything is fully polished and tested. This makes sense in the physical world, but can actually be detrimental in the software world.

Why is it important to use an iterative approach to software development? Since deploying software can be done quickly and changes can be rapidly fixed if necessary, frequent software releases allow customers to receive updates more quickly than if they waited for a major release. This allows customers to provide feedback if the features are not useful or if they work improperly. This feedback creates a loop that allows developers to go back and ‘get it right’ if they need to in a more controlled environment. Without this loop, it’s possible to spend months developing features that don’t actually meet user requirements or to create bugs that require substantial cost to find and rectify. This means wasted development time as well as missed opportunity costs. Additionally, small frequent software updates means that each individual update can be tested independent of other changes and issues are kept much smaller than in large updates. Frequent software updates also means a decreased time-to-market with new features.

Iterative approaches to software improve user experience, improve time-to-market, decrease difficult to find bugs, and shorten and simplify test cycles. Are you frequently releasing software updates, or are you treating software like it’s a physical commodity?

Posted on

Loosely Coupled Systems

One of the principles of modern engineering is to create loosely coupled systems. But what does that mean and why is it so important?

In the past, it was common to create huge systems that included code for a wide variety of different functions. For example, an eCommerce system may include code for accessing the database, processing credit card payments, and interacting with an inventory management system. While having all the code in one place may sound great, it has some serious drawbacks. For example, maintenance becomes increasingly difficult the larger an application becomes. Additionally, testing must include the entire system, and deployment is an all or nothing deal. Upgrades are also more difficult as the entire system must be upgraded at once and opportunities for regression errors multiply.

Today, systems strive to be modular and loosely coupled. One piece may do credit card processing, another service may provide database access, and still another service may provide email support. While there are more pieces, these pieces can be assembled into a wide variety of configurations across different applications and these modules can be more easily tested. Once the credit card service is deployed, for example, it does not need to be changed or tested again until new features are required or bugs are found. Each piece can use the best technology for the task, and upgrades can happen on a per-service basis.

Currently, these services are often deployed as JSON-based REST services. These types of services are now becoming ubiquitous. A large variety of publicly available services are available for things like weather data, stock quotes, ISO country codes, etc.  This modular approach not only decreases development effort, it improves application stability as well.

On any project you’re involved in, whether it’s in writing the code or managing the project, modularity and loose coupling should be one of the most important guiding principles.

Posted on

Minimum Viable Product

An important, but often ignored, concept in the realm of software development is the notion of Minimum Viable Product. Defining a Minimum Viable Product (or MVP) provides several benefits for software development teams and companies. First, let’s define Minimum Viable Product. The MVP is simply the absolute minimum requirements for a working product. For instance, if you’re writing a calculator application for a mobile device, the MVP would simply add, subtract, multiply, and divide numbers. It would not, however, need to perform more complicated functions such as square root, exponents, trig functions, averaging, etc. These features would be added to future versions if necessary.

What makes this idea so important?  When we define the Minimum Viable Product, we can easily accomplish several things. First, when we eliminate all the fluff, we can focus on core functionality and release a product quickly. This reduces time to market and increases revenue generation from the application. Second, creating a functional application more quickly allows decision makers within the organization to rapidly determine if the software meets the necessary requirements that the product was intended to solve. This ability for management to be involved earlier in the process ensures that development resources don’t end up heading down the wrong path. Third, deploying the MVP early ensures that customers and end users can interact and provide feedback early in the process when it’s far easier (and cheaper) to change direction instead of waiting for a months or years long process.

Unfortunately, many organizations insist that every feature is essential to the application which ensures long development cycles and dramatically longer time-to-market. Meanwhile, more agile teams are capturing marketshare before you.

When thinking of a new product, software application, tool, or mobile application, the first objective is to define the MVP so that you can quickly create a prototype, verify functionality, ensure essential functionality is present, and then get your software in the hands of users. Only then do you circle back for round two where you begin to add new features by starting with the most important tasks. This process will not only improve your marketshare, but it will also help ensure that the products and services being created actually bring value instead of spending years on software projects that bring no measurable value.

Posted on

Work in Progress

When most people think about DevOps, Continuous Integration (CI), or Continuous Deployment (CD) they think about tools like Jenkins or Bamboo. And, indeed, these tools are indispensable for rapidly moving software through the development pipeline. However, an even more rudimentary principle is required to truly enjoy the benefits of any CI/CD environment – limiting work in progress (WIP). Work in progress is the number of items that are currently being developed. The higher the number, the more difficult and time consuming deployments become. And, conversely, the lower of number of features in development, the lower the risk for the deployment and the more rapidly it can be deployed to live systems. Limiting the WIP to a small number – a single new feature or a couple of important bug fixes – ensures that you can get the changes tested and deployed in short order. This goes well with the ideals of agile development – quick sprints where a small number of items are completed – and ensures that focus is kept on the most important items. Once that sprint is complete, a new version can be pushed to end users. This ensures users have want they want, turnaround time is low, and bugs are kept to a minimum as changes are small and easily tested. This principle applies well beyond software. In fact, it was originally documented on factory floors. Creating products in small batches ensures that if there is a problem, it’s limited to a small batch and that the first shippable products are available more quickly than large batch processing. How does your company handle work in progress? A small number of items, or huge batches? Does it take forever to release a new version or to have a product ready to market? Consider the impact of limiting the work in progress to a small number and frequently releasing your software or product and how it could impact your time-to-market and ultimately increase revenue!

Posted on

Flavor of the Week

Ice Cream

The internet has had an amazing impact on the world. Not only does it allow us access to more information faster than ever possible before, but it also allows us to have access to software and libraries instantly. As a developer, I rely on tools like Maven or Gradle to automatically retrieve code libraries from the internet on demand. But this speed of disseminating information and code has also created a ‘flavor of the week’ environment for programming frameworks and technologies.  Twenty years ago, if you were a professional developer you probably worked in C, C++, Visual Basic, or Delphi. Today, it’s hard to count all the different languages out there. In the Java world alone we have Java, Scala, Groovy, Clojure, Kotlin, and others. For JavaScript frameworks, developers can choose from jQuery, REACT, and Angular – which just accounts for the most popular options. With each new project, companies examine the tools available and change their development stack to utilize the latest and greatest. What used to take a decade or more to become legacy code is now obsoleted in years.

When I entered the IT world, I needed to know one programming language – that was it. Now, looking at the technical requirements provided by employers is a daunting task as each company has cherrypicked what they believe to be the best technologies in a variety of different realms. Developers are increasingly expected to be experts of an unimaginable number of technologies. What’s worse, few technologies never reach maturity before they’re completely overhauled. Consider, for instance, Angular. While it’s an excellent framework for application development, it’s nearly impossible to find answers to questions since the framework has changed so much since its creation. Long-term, the IT world is going to end up with innumerable projects that are no longer maintainable because their technology stack has become obsolete or developers have moved on to the new flavor of the week.  Even worse, programmers are becoming a jack-of-all-trades which means that code quality is poorer, bugs increase, and time-to-market is increased as nobody has a mastery of anything they’re working with.

None of this is bringing any value to end users or tech companies. In fact, it’s just creating more code that will be obsolete before it’s even deployed.

Proudly powered by WordPress