One of my favorite technologies is Microsoft’s .Net Core 2.0 framework. Some of the things I love are that it’s cross-platform, it has an excellent framework for interacting with databases, and their MVC model is straight forward to implement. However, with all that Microsoft has managed to accomplish, there are still some major shortcomings. (Before going forward, let me point out that I develop on a Mac using Visual Studio – so a native Windows user may have a different experience.) For example, while the Entity Framework is amazing – the support for stored procedures is less stellar. After days of searching for solutions, I finally came across a working answer. This is the biggest problem with .Net Core – lack of answers on the web. Or more properly, the lack of the right answer. Look up any question on how to do something using .Net Core and you’ll find a dozen different answers – none of which actually work. This problem was repeated over and over again. When I tried to store a simple session variable, I saw tutorial after tutorial – none of them worked. Next on the list is the NuGet package manager. As a Java user, I have grown accustomed to pulling libraries from a remote repository using Maven or Gradle – so I was happy to see similar functionality for .Net. The problem? It’s impossible to find what package contains the thing you’re looking for. Online code never includes the appropriate imports, so you’re left to guess as to what you need. This problem is exacerbated by the horrible idea of creating class extensions. Every package you import can add extensions to classes in other packages. This sounds great, but in reality it only complicates figuring out what to import. Last, the way you configure the web server is the worst I have ever seen – you configure it in code! Want to use a different port? Hard code it. Need to run the app in a subdirectory? Hard code it. Need to setup database support? Hard code it. So, as the app goes from machine to machine it actually has to be rewritten to support the configuration required on the new server. I have seen various examples of how to use configuration files to accomplish this, but like everything else the documentation is poor and inconsistent at best. In the end, while I do love what Microsoft has accomplished with .Net Core 2.0 and their MVC framework, it still has a long way to go before I would consider it to be able to seriously complete with more mature frameworks.
Web Severs
Earlier this week, I posted about the importance of analyzing requirements to determine the best technology for solving a given problem. Today, I want to dive deeper and talk about different web servers and where they best fit. Having spent well over a decade developing Java web applications, I’m often quick to assume JBoss or Tomcat is the best answer. But as this article will point out, that’s often simply not the case. (Note: for this article, I am not concerning myself with the difference between a web server and an application server. At the end of the day, if the solution solves the problem, who cares?) First on our list of servers is Apache. Apache has been around since 1995, and is often the first choice for websites. If you run a Linux machine, chances are you already have Apache installed. The real power of Apache lies in the modules you can use to add additional functionality to the server. Two of the most significant enable PHP support and running a reverse proxy. Apache with PHP would be a good choice for a WordPress site, or for eCommerce sites running PHP. However, as a site gets larger, PHP becomes more difficult to manage than other solutions. With reverse proxy support enabled, you’ll often see Apache setup to run in front of other servers – such as JBoss or Tomcat – or to enable load balancing. Speaking of JBoss and Tomcat, these are both excellent servers on their own. Both are intended to run Java web applications which – due to the massive number of available Java libraries – can do just about anything. However, all that power comes with a cost. JBoss and Tomcat servers and applications are not particularly lightweight or easy to setup, and the Java code for many common tasks will be far more complex than other solutions. For example, what if your application makes heavy use of JSON and REST services? While Java will handle this very well, a simple Node.js server may be much more lightweight. I can create a REST server in JavaScript in a fraction of the time of creating a corresponding Java server – regardless of the Java technologies used. Maybe you want all the features of JBoss or Tomcat, but you want to deploy to Microsoft’s Azure platform. Microsoft’s .NET Core platform is an excellent choice. Not only is .NET Core now cross platform (running on Linux and Mac), but their MVC framework is amazing. Furthermore, their entity framework is probably the best of any I have ever used. What about solutions for makers and tech hackers? Python has several server implementations including Flask and Django. Not only is it an excellent language for beginners, but it’s become one of the primary languages used for developing on the Raspberry Pi as well as other hardware platforms. Countless other servers could be mentioned, but it’s easy to see from the above that solutions exist for every possible web server scenario. So, when you go to chose your platform – ask yourself what platform best fits your needs. Find an engineer to analyze the problem and provide recommendations on best technologies. At the end of the day, selecting the right technology will result in a much shorter time to market and substantially lower development costs.
Requirements Analysis
One of the most important skills for a developer is the ability to analyze requirements and determine the most appropriate solution. For many developers, this means they will determine the best suite of tools that fits within their preferred development environment. For example, a Java developer may decide if JDBC or JPA is a better option for connecting to their database. Note, it was already assumed that the application would be written in Java – other options like C# or PHP were ignored because the developer making the choice was a Java developer. The problem is that there may be better options depending on the requirements of the project. For example, I am currently working on a mobile project that uses a JavaScript framework. One of the requirements of the app is to create a fairly detailed PDF document. This document was created using pdfmake – an excellent toolkit for making PDF documents based on a simple configuration file. However, as the project grew, it was requested to have a web service written that would be given the configuration, generate the PDF, and send an email. What server language would I use? I could use Java – but do I want to rewrite the entire PDF generator in a new language? Absolutely not. What else could work? In the end, I opted to use a Node.js solution. Why? Because it would be utterly trivial to use the existing JavaScript PDF code within a Node application. In fact, I managed to write all the necessary functionality in a single file with less than 100 lines of code. Had I selected Java, it would have easily grown into several dozen file, hundreds of lines of code, and substantially more billable hours.
Unfortunately, technology decisions are made every single day by organizations that insist on a language or framework before the requirements are even known. Better options may exist, but lack of knowledge of competing technologies prevents their selection. In the end, projects take longer to develop, cost more, and become increasingly difficult to maintain. Certainly no developer can be an expert in every technology, but any more senior developer should be able to provide a variety of competing solutions to any problem as well as indicate the pros and cons of each. When the best technology is selected, projects come in ahead of schedule and under budget. Time-to-market is decreased, maintenance costs are minimized, and – in the end – the organization benefits.
Text File Parsing
It’s no secret to anyone that I’m a Unix guy. I’ve worked on Unix-based systems for over 20 years now. If I’m forced to use a Windows machine, the first thing I do is install software to provide me a more Unix-like experience. I could give dozens of reasons why I love Unix such as programming environments, robust command line utilities, stability, etc. But one of the most useful to me is the ability to perform complex text manipulations from the command line. Parsing log files, editing text files on the fly, creating reports, all easy from a Unix command line. Of all the commands I use, some of my favorites are sed, tr, grep, and cut. With just those four commands I can make magic happen. For instance, today I had a configuration file for a git-o-lite server that I needed to use to generate a list of repositories from. I could open the file in a text editor and edit it… but that’s work. As a programmer, I program things so I don’t have to do trivial work. Besides, given the large number of repositories in the config file it would take too much time and be prone to error. Knowing the format of the config file, I opened up a command prompt and started stringing together commands to transform the data one step at a time. At the end, I had a list of every repository nicely sorted alphabetically. This list could then be fed into another process that might perform some maintenance task on each repository or perform some other task. In the end, my command was a long string of small commands strung together.
cat gitolite.conf | grep 'repos/' | cut -d'=' -f2 | tr ' ' '\n' | tr -d ' ' | grep -v '^$' | grep -v '@' | grep 'repos' | sort
While it may look cryptic to the uninitiated, I’ll explain each command and why it was used. First, I used the cat file to display the contents of the config file for git-o-lite. The cat command is useful for displaying text data from a command prompt in Unix, and is almost always the starting point for any Unix text-processing. Next, the grep command is executed to find all lines containing the text ‘repos/’ which I know is the starting point for all the repository names in the configuration file. Grep is another commonly used Unix command that is used to search a file for a text string and display matching rows. Numerous versions of grep exist providing all kinds of advanced functionality, but basic grep is still the most commonly used. Now that I have all lines containing repository names, I can begin to process that list. I start by using cut to remove the variable names. Since Git-o-lite allows variables to defined (@var = value), and I only want the value, I will tell cut to split on the equal sign delimiter (-d’=’) and give me only the second field (-f2). Since multiple repository names may be on a single line, I next need to split the data on the space so that each repository is on it’s own line. The tr command will translate one character to another. So, in this instance, I will change ‘ ‘ to ‘\n’ (\n is a newline character – like hitting the return key on the keyboard). Next, I delete any remaining spaces using using the -d flag for the tr command. At this point, my output contains empty lines which I want to remove. The -v argument for grep will remove lines containing the supplied search string. Here, the cryptic ‘^$’ is used where ^ is the beginning of the line and $ is the end of the line (these are standard grep patterns). Next, I run through a few more grep commands to cleanup the data and then pipe the content to the sort function. Now, I have my list of repositories ready for whatever follow on processing I want. Last step, copy the above commands to a shell script so that I don’t have to type all those commands in again.
Throughout my career, I have used the above process innumerable times. I can extract any manner of text data from a file quickly and easily simply by using a string of commands to incrementally transform the data to my desired output. If you are a programmer, and you’re not familiar with the above commands, you’re probably working too hard.
LaTeX?
Numerous times throughout my life I have wanted to write a book. I even have several manuscripts on my computer in various stages of completion. During the last month, I decided to try again. Step one was to decide what I wanted to write about. So, I selected about 10 topics and chose one for my first book. Step two, determine what tool I would use to write my book. A variety of software applications are available including Microsoft Word, Apple Pages, and Adobe InDesign. But one of my requirements is that the format be text-based so that I can easily store the document as well as the history in my Git repository. Initially, I thought that Markdown would be a great format – text-based, easy to store in a source repository, and no special software required. However, after a short amount of work, I realized that Markdown didn’t really support the things I would want for writing. The first issue was with inadequate support for page layouts including images. Other issues followed, and I went back to the drawing board searching for a better alternative. While searching the internet, I found numerous sites recommending LaTeX. I never used LaTeX before, but I do remember seeing it referenced throughout the years on Linux systems. Could this be the answer? Does it meet my requirements? LaTeX is similar to HTML or Markdown. Character sequences can be used to indicate things like the table of contents, a chapter, a section, etc. Additional functionality can be added by using packages to fill all manner of typography needs. The files are text, and can easily be managed by a source control system and edited from any standard text editor. What about the output? I can easily run the LaTeX commands to output my document in PDF format, and I have even read that tools exist for conversion to ePub. When I add new chapters or update existing content, I can have my Jenkins server automatically build the full document and email it to me or publish in any why I choose. While I have only been tinkering with LaTeX for a short time, I have been very impressed with what I’ve been able to accomplish so far and intend to master this amazing tool to create documents and manuscripts.
What is a Maker?
Last week, I went to lunch with a business colleague. As we were talking, I mentioned that I was in the midst of writing a book targeting makers. “What’s a maker?” He asked. After pondering the question for a few seconds, I realized it was actually a really good question! What is a maker? I could say that it’s people who tinker with hardware platforms such as Arduino or Raspberry Pi, but that’s a rather narrow definition. What about the man who builds an aquaponics system? Is he a maker too? Or the girl who knits hats, is she a maker? Does her knit hat need to include electronics for inclusion in the realm of makers? Placing a definition on ‘maker’ is actually harder than it looks. However, if you subscribe to Make Magazine or ever visit a Makerfaire, you will learn pretty quick that the term “maker” is rather broad. A few years ago, I saw makers blowing glass, forging swords, and knitting blankets – hardly ‘tech savvy’ projects, but still makers. Of course, I saw countless tech projects such as Raspberry Pi clusters and IoT devices too as well as jewelry makers using circuit boards, artists drawing robots, and things made of Legos. So what is a maker? The best answer I can come up with is that a maker is someone who uses the tools and materials around him or her to make something useful or even just novel. It’s being part of a movement that empowers people to solve problems on their own. Makers are the people you want to be with when the world ends – because they’ll have the tools and knowledge to rebuild society. Makers are the people who caused the renaissance – great minds like Leonardo DeVinci. Makers are the jack-of-all-trades men and women who can program a micro controller, 3D print a case, and use it for the robot they cut, welded, and painted themselves. As a business, why should you care? What difference does it make to you? Makers are the people in your organization that will solve the problems to move your business to the next level. They are the men and women who poke something to see how it works and how they can improve it. They are the problem solvers you want on every team in your organization because they are the thinkers that will create the great things of tomorrow!
Passion
Recently, I read an article on LinkedIn questioning the notion that good developers are those who write code outside of business hours – not for their job, but for their own interests. The article suggested that you could be a great developer and go home and pursue other endeavors – that it was unfair to expect that your best developers are writing code at home. I could not disagree more. Sure, a person can be a competent developer and only write code during business hours. But I assure you that such a developer will never be considered a “rock star” programmer. Those who are the best in their field are always those with passion – and passion is not a 9-5 job. The article tried to compare programming to carpentry. It asked if we would expect a carpenter to be engaged in woodworking at home to to be considered competent. That seems silly to me. If I’m looking for someone to create a complex piece of wooden cabinetry with hand carvings, I can assure you I’m looking for an artisan with a passion for his work – one who has honed that skill through years of hard work outside his 9-5 job. I have a good friend who is a musician – a highly skilled performer who makes everyone he plays with sound better because of the mastery of his art. Music is his passion. He didn’t get that skill working 9-5 – it’s what he loves and it shows when you hear him perform. Listen to any business leader talk and what do they all talk about? Passion. It’s passion that brings forth the best in humanity. It’s a deep love of what we do that allows us to become a rock star in our field. In every corner of life, those with passion are the ones we remember – Steve Jobs, Mother Theresa, Grandi, Tom Brady – people passionate about their work. Want to be remembered by history and admired by your peers? Be passionate about what you do and remember that passion is not a 9-5 job.
Workplace Efficiency
All businesses share the same basic objective – to make money. That’s pretty simple, but what enables that to happen? Business leaders focus on increasing revenue, decreasing costs, increasing efficiency, ensuring compliance with regulatory guidelines, and finding and exploiting any competitive advantages they have. Of all the above objectives, one of the easiest is to increase efficiency and remove inefficiencies. Most people hate bureaucracies – we see them when we look to the federal government, the DMV and, all too often, the very companies we work for. Bureaucracies make us feel powerless and unimportant. They beat us down with rules and restrictions that keep us from accomplishing our job. Bureaucracies are the very antithesis of efficiency. In your business, are policies and procedures intended to chain your employees down, or liberate them to do the work you pay them to do? Are your processes streamlined for the mission at hand, or dictated by bureaucrats above in such a way as to slow down your business? As businesses grow, they tend to become more rigid and bureaucratic – more layers of management, more company ‘yes men’ who won’t challenge the status quo, and more workers who show up and punch the clock instead of having any excitement for the mission of the organization. Ever work at a small businesses? The focus is always on overtaking the Goliath in the market. Policies and procedures that prevent that from happening are removed, workers are empowered to do their best. People have a shared sense of purpose, and often develop a feeling of family. What can we learn from these observations? First, if you’re a leader within a large organization, how can you remove red tape and empower your people? What do you need to do to ensure that your people are working at their best? Are you creating roadblocks to your employees’ work? Employees want to feel empowered – they want to feel that their leaders are on their side. They want the bureaucracy removed so they can do their best. Second, if you work in a small business, recognize that your lack of rigidity may be the single biggest competitive advantage you have. With no bureaucracy to hold you back, you can find the best way to accomplish the mission and execute without roadblocks. Empower your team to win and then step back and watch them achieve. As a small business, your best way to attack Goliath is through superior processes, quicker turn around, less paperwork, and having employees that feel empowered to accomplish the mission. No business owner ever said “I think the federal government would be a great model for how to run my business!” but that is exactly what happens to businesses all across our nation. It’s time to change that trend!
Professor Qualifications
I recently saw a job post for a computer programming instructor. The post required that the applicant have a masters degree as well as four years of professional experience. I was a little shocked by this. Does four years of experience in programming really qualify someone to teach others? To become a Master Electrician in New York takes a minimum of 7 and a half years of experience. To become a Master instructor in Taekwondo can easily take more than 10 years in most organizations. Mastery of most tasks takes a substantial amount of time – and programming is no different. To make it more difficult, programming projects tend to be long-term – often taking a year or more just for the initial release. This means that during a four year term of employment with a company, a programmer may only actually work on a very small number of projects. When I think of a teacher, I think of someone with a breadth and depth of knowledge on the task. Someone who has mastered all tasks associated with the subject they are teaching. But after four years, I don’t believe that any programmer could be said to be a master. In fact, in most organizations, titles like senior engineer would be reserved for developers with far more than four years of experience. Why does any of this matter? It matters because it shows part of the problem we have finding qualified programmers to fill positions. After all, how can we expect students coming out of our universities to have any meaningful skill in development if their instructors never ascended above the rank of junior developer? How can someone who never made any meaningful contributions to the architecture of an application mentor others on how to do so? Instructors should be those most qualified among our engineers who have spent decades writing code – men and women who can tell you about several languages, talk about the evolution of programming through their own personal experience, and talk about how they solved a wide variety of problems. Instructors should have a portfolio of applications and organizations they have impacted and understand software engineering, computer architecture, and all aspects of the software development lifecycle. If we settle for less in our teachers we are setting our students up for failure and ensuring that we will continue to see more tech jobs outsourced to countries that can do a better job educating the next generation of developers.
Perfection Paralysis
A major problem with many projects is the quest for perfection. We don’t want to release a software application until it looks perfect, there are no bugs, it’s fast, and the user experience is flawless. And that’s a great ideal for any project. Unfortunately, the quest for perfection invariably adds scope-creep. What was originally deemed a good design is now less than perfect and requires tweaks. The performance is just not fast enough yet, so we’ll tune the servers. An executive decided there is a new piece of required functionality that simply must go in. On and on we go, and the app release is pushed back further and further. Projects can go on for a year or more before being released. Certainly we want solid software applications, but what does this quest for perfection bring us? In the end, very little of value. As release dates get pushed back, the customer begins to lose faith in the abilities of the development team to deliver. Increased time-to-market means we miss opportunities to sell our product. Increased release cycle times make us less responsive to user issues we didn’t see. These problems are among the issues solved by agile development. Frequently releasing code means that users can start getting essential functionality quicker, time-to-market is diminished, and teams can react to changing user requirements quickly. But to get there, we have to determine what the minimum viable product is and accept that things may not be perfect. We need to accept that our initial version will not be the best there is, but that it’s good enough to do what we set out to do. Understandably, this may feel uncomfortable to the perfectionist in us – after all, we want to give our absolute best. We want a product that we can be proud of and that the customer is dying to have. But if we wait, with delay after delay, we run the risk of having the best app that will never be used because someone else beat you to it.