Talixa Tech Blog

March 2, 2018

Text File Parsing

It’s no secret to anyone that I’m a Unix guy. I’ve worked on Unix-based systems for over 20 years now. If I’m forced to use a Windows machine, the first thing I do is install software to provide me a more Unix-like experience. I could give dozens of reasons why I love Unix such as programming environments, robust command line utilities, stability, etc. But one of the most useful to me is the ability to perform complex text manipulations from the command line. Parsing log files, editing text files on the fly, creating reports, all easy from a Unix command line. Of all the commands I use, some of my favorites are sed, tr, grep, and cut. With just those four commands I can make magic happen. For instance, today I had a configuration file for a git-o-lite server that I needed to use to generate a list of repositories from. I could open the file in a text editor and edit it… but that’s work. As a programmer, I program things so I don’t have to do trivial work. Besides, given the large number of repositories in the config file it would take too much time and be prone to error. Knowing the format of the config file, I opened up a command prompt and started stringing together commands to transform the data one step at a time. At the end, I had a list of every repository nicely sorted alphabetically. This list could then be fed into another process that might perform some maintenance task on each repository or perform some other task. In the end, my command was a long string of small commands strung together.

cat gitolite.conf | grep 'repos/' | cut -d'=' -f2 | tr ' ' '\n' | tr -d ' ' | grep -v '^$' | grep -v '@' | grep 'repos' | sort

While it may look cryptic to the uninitiated, I’ll explain each command and why it was used. First, I used the cat file to display the contents of the config file for git-o-lite. The cat command is useful for displaying text data from a command prompt in Unix, and is almost always the starting point for any Unix text-processing. Next, the grep command is executed to find all lines containing the text ‘repos/’ which I know is the starting point for all the repository names in the configuration file. Grep is another commonly used Unix command that is used to search a file for a text string and display matching rows. Numerous versions of grep exist providing all kinds of advanced functionality, but basic grep is still the most commonly used. Now that I have all lines containing repository names, I can begin to process that list. I start by using cut to remove the variable names. Since Git-o-lite allows variables to defined (@var = value), and I only want the value, I will tell cut to split on the equal sign delimiter (-d’=’) and give me only the second field (-f2). Since multiple repository names may be on a single line, I next need to split the data on the space so that each repository is on it’s own line. The tr command will translate one character to another. So, in this instance, I will change ‘ ‘ to ‘\n’ (\n is a newline character – like hitting the return key on the keyboard). Next, I delete any remaining spaces using using the -d flag for the tr command. At this point, my output contains empty lines which I want to remove. The -v argument for grep will remove lines containing the supplied search string. Here, the cryptic ‘^$’ is used where ^ is the beginning of the line and $ is the end of the line (these are standard grep patterns). Next, I run through a few more grep commands to cleanup the data and then pipe the content to the sort function. Now, I have my list of repositories ready for whatever follow on processing I want. Last step, copy the above commands to a shell script so that I don’t have to type all those commands in again.

Throughout my career, I have used the above process innumerable times. I can extract any manner of text data from a file quickly and easily simply by using a string of commands to incrementally transform the data to my desired output. If you are a programmer, and you’re not familiar with the above commands, you’re probably working too hard.

March 1, 2018March 1, 2018

LaTeX?

Numerous times throughout my life I have wanted to write a book. I even have several manuscripts on my computer in various stages of completion. During the last month, I decided to try again. Step one was to decide what I wanted to write about. So, I selected about 10 topics and chose one for my first book. Step two, determine what tool I would use to write my book. A variety of software applications are available including Microsoft Word, Apple Pages, and Adobe InDesign. But one of my requirements is that the format be text-based so that I can easily store the document as well as the history in my Git repository. Initially, I thought that Markdown would be a great format – text-based, easy to store in a source repository, and no special software required. However, after a short amount of work, I realized that Markdown didn’t really support the things I would want for writing. The first issue was with inadequate support for page layouts including images. Other issues followed, and I went back to the drawing board searching for a better alternative. While searching the internet, I found numerous sites recommending LaTeX. I never used LaTeX before, but I do remember seeing it referenced throughout the years on Linux systems. Could this be the answer? Does it meet my requirements? LaTeX is similar to HTML or Markdown. Character sequences can be used to indicate things like the table of contents, a chapter, a section, etc. Additional functionality can be added by using packages to fill all manner of typography needs. The files are text, and can easily be managed by a source control system and edited from any standard text editor. What about the output? I can easily run the LaTeX commands to output my document in PDF format, and I have even read that tools exist for conversion to ePub. When I add new chapters or update existing content, I can have my Jenkins server automatically build the full document and email it to me or publish in any why I choose. While I have only been tinkering with LaTeX for a short time, I have been very impressed with what I’ve been able to accomplish so far and intend to master this amazing tool to create documents and manuscripts.

February 26, 2018July 14, 2019

What is a Maker?

Last week, I went to lunch with a business colleague. As we were talking, I mentioned that I was in the midst of writing a book targeting makers. “What’s a maker?” He asked. After pondering the question for a few seconds, I realized it was actually a really good question! What is a maker? I could say that it’s people who tinker with hardware platforms such as Arduino or Raspberry Pi, but that’s a rather narrow definition. What about the man who builds an aquaponics system? Is he a maker too? Or the girl who knits hats, is she a maker? Does her knit hat need to include electronics for inclusion in the realm of makers? Placing a definition on ‘maker’ is actually harder than it looks. However, if you subscribe to Make Magazine or ever visit a Makerfaire, you will learn pretty quick that the term “maker” is rather broad. A few years ago, I saw makers blowing glass, forging swords, and knitting blankets – hardly ‘tech savvy’ projects, but still makers. Of course, I saw countless tech projects such as Raspberry Pi clusters and IoT devices too as well as jewelry makers using circuit boards, artists drawing robots, and things made of Legos. So what is a maker? The best answer I can come up with is that a maker is someone who uses the tools and materials around him or her to make something useful or even just novel. It’s being part of a movement that empowers people to solve problems on their own. Makers are the people you want to be with when the world ends – because they’ll have the tools and knowledge to rebuild society. Makers are the people who caused the renaissance – great minds like Leonardo DeVinci. Makers are the jack-of-all-trades men and women who can program a micro controller, 3D print a case, and use it for the robot they cut, welded, and painted themselves. As a business, why should you care? What difference does it make to you? Makers are the people in your organization that will solve the problems to move your business to the next level. They are the men and women who poke something to see how it works and how they can improve it. They are the problem solvers you want on every team in your organization because they are the thinkers that will create the great things of tomorrow!

February 22, 2018

“Revolutionary” Tech

In 1974, a Hungarian invented one of the most popular and influential toys of all time, the Rubik’s cube. Considering that I was born well after that time, I can safely say that I cannot remember a time without this toy. Growing up, however, I watched the Rubik’s company try to make toy after toy in some way related to the original, but they always tried new and “innovative” renditions.

Since my family members always knew how big of a Rubik’s cube fan I am, I would get all kinds of these newer toys as birthday and Christmas gifts. While many of them were flashy and initially interesting, I ultimately grew tired of them and would go back to playing with the original. After all, the original was fascinating… captivating even, and I was determined to learn how to solve it.

After spending a considerable amount of time experimenting and playing with my cube, one of the teachers at my elementary school gave me a list of algorithms showing how to manipulate individual squares in very specific ways. The summer after, I accomplished my goal to learn to solve the cube, but I was still hungry for more.

While my first few solves averaged around 10 minutes, I wanted to continue to get better and faster. I studied and fine tuned my process of solving the cube, realizing that at a certain point the design of the cube itself was holding me back. I then looked into what other cubes were out there and how they were designed to be more efficient and effective. These days I solve the cube in under a minute, and the best people in the world can solve it in a matter of seconds.

Constantly evaluating and enhancing processes and expanding resources also holds value when thinking about technology and business. Businesses that are growing very rapidly today focus their time, energy, and resources on making gradual improvements, becoming more resourceful. By learning from this, organizations can ensure that they are effectively working toward their goals, accomplishing more by using the most effective resources to the best of their ability.

February 19, 2018October 27, 2019

Passion

Recently, I read an article on LinkedIn questioning the notion that good developers are those who write code outside of business hours – not for their job, but for their own interests. The article suggested that you could be a great developer and go home and pursue other endeavors – that it was unfair to expect that your best developers are writing code at home. I could not disagree more. Sure, a person can be a competent developer and only write code during business hours. But I assure you that such a developer will never be considered a “rock star” programmer. Those who are the best in their field are always those with passion – and passion is not a 9-5 job. The article tried to compare programming to carpentry. It asked if we would expect a carpenter to be engaged in woodworking at home to to be considered competent. That seems silly to me. If I’m looking for someone to create a complex piece of wooden cabinetry with hand carvings, I can assure you I’m looking for an artisan with a passion for his work – one who has honed that skill through years of hard work outside his 9-5 job. I have a good friend who is a musician – a highly skilled performer who makes everyone he plays with sound better because of the mastery of his art. Music is his passion. He didn’t get that skill working 9-5 – it’s what he loves and it shows when you hear him perform. Listen to any business leader talk and what do they all talk about? Passion. It’s passion that brings forth the best in humanity. It’s a deep love of what we do that allows us to become a rock star in our field. In every corner of life, those with passion are the ones we remember – Steve Jobs, Mother Theresa, Grandi, Tom Brady – people passionate about their work. Want to be remembered by history and admired by your peers? Be passionate about what you do and remember that passion is not a 9-5 job.

February 16, 2018

Workplace Efficiency

All businesses share the same basic objective – to make money. That’s pretty simple, but what enables that to happen? Business leaders focus on increasing revenue, decreasing costs, increasing efficiency, ensuring compliance with regulatory guidelines, and finding and exploiting any competitive advantages they have. Of all the above objectives, one of the easiest is to increase efficiency and remove inefficiencies. Most people hate bureaucracies – we see them when we look to the federal government, the DMV and, all too often, the very companies we work for. Bureaucracies make us feel powerless and unimportant. They beat us down with rules and restrictions that keep us from accomplishing our job. Bureaucracies are the very antithesis of efficiency. In your business, are policies and procedures intended to chain your employees down, or liberate them to do the work you pay them to do? Are your processes streamlined for the mission at hand, or dictated by bureaucrats above in such a way as to slow down your business? As businesses grow, they tend to become more rigid and bureaucratic – more layers of management, more company ‘yes men’ who won’t challenge the status quo, and more workers who show up and punch the clock instead of having any excitement for the mission of the organization. Ever work at a small businesses? The focus is always on overtaking the Goliath in the market. Policies and procedures that prevent that from happening are removed, workers are empowered to do their best. People have a shared sense of purpose, and often develop a feeling of family. What can we learn from these observations? First, if you’re a leader within a large organization, how can you remove red tape and empower your people? What do you need to do to ensure that your people are working at their best? Are you creating roadblocks to your employees’ work? Employees want to feel empowered – they want to feel that their leaders are on their side. They want the bureaucracy removed so they can do their best. Second, if you work in a small business, recognize that your lack of rigidity may be the single biggest competitive advantage you have. With no bureaucracy to hold you back, you can find the best way to accomplish the mission and execute without roadblocks. Empower your team to win and then step back and watch them achieve. As a small business, your best way to attack Goliath is through superior processes, quicker turn around, less paperwork, and having employees that feel empowered to accomplish the mission. No business owner ever said “I think the federal government would be a great model for how to run my business!” but that is exactly what happens to businesses all across our nation. It’s time to change that trend!

February 13, 2018November 24, 2019

Professor Qualifications

I recently saw a job post for a computer programming instructor. The post required that the applicant have a masters degree as well as four years of professional experience. I was a little shocked by this. Does four years of experience in programming really qualify someone to teach others? To become a Master Electrician in New York takes a minimum of 7 and a half years of experience. To become a Master instructor in Taekwondo can easily take more than 10 years in most organizations. Mastery of most tasks takes a substantial amount of time – and programming is no different. To make it more difficult, programming projects tend to be long-term – often taking a year or more just for the initial release. This means that during a four year term of employment with a company, a programmer may only actually work on a very small number of projects. When I think of a teacher, I think of someone with a breadth and depth of knowledge on the task. Someone who has mastered all tasks associated with the subject they are teaching. But after four years, I don’t believe that any programmer could be said to be a master. In fact, in most organizations, titles like senior engineer would be reserved for developers with far more than four years of experience. Why does any of this matter? It matters because it shows part of the problem we have finding qualified programmers to fill positions. After all, how can we expect students coming out of our universities to have any meaningful skill in development if their instructors never ascended above the rank of junior developer? How can someone who never made any meaningful contributions to the architecture of an application mentor others on how to do so? Instructors should be those most qualified among our engineers who have spent decades writing code – men and women who can tell you about several languages, talk about the evolution of programming through their own personal experience, and talk about how they solved a wide variety of problems. Instructors should have a portfolio of applications and organizations they have impacted and understand software engineering, computer architecture, and all aspects of the software development lifecycle. If we settle for less in our teachers we are setting our students up for failure and ensuring that we will continue to see more tech jobs outsourced to countries that can do a better job educating the next generation of developers.

February 11, 2018February 11, 2018

Perfection Paralysis

A major problem with many projects is the quest for perfection. We don’t want to release a software application until it looks perfect, there are no bugs, it’s fast, and the user experience is flawless. And that’s a great ideal for any project. Unfortunately, the quest for perfection invariably adds scope-creep. What was originally deemed a good design is now less than perfect and requires tweaks. The performance is just not fast enough yet, so we’ll tune the servers. An executive decided there is a new piece of required functionality that simply must go in. On and on we go, and the app release is pushed back further and further. Projects can go on for a year or more before being released. Certainly we want solid software applications, but what does this quest for perfection bring us? In the end, very little of value. As release dates get pushed back, the customer begins to lose faith in the abilities of the development team to deliver. Increased time-to-market means we miss opportunities to sell our product. Increased release cycle times make us less responsive to user issues we didn’t see. These problems are among the issues solved by agile development. Frequently releasing code means that users can start getting essential functionality quicker, time-to-market is diminished, and teams can react to changing user requirements quickly. But to get there, we have to determine what the minimum viable product is and accept that things may not be perfect. We need to accept that our initial version will not be the best there is, but that it’s good enough to do what we set out to do. Understandably, this may feel uncomfortable to the perfectionist in us – after all, we want to give our absolute best. We want a product that we can be proud of and that the customer is dying to have. But if we wait, with delay after delay, we run the risk of having the best app that will never be used because someone else beat you to it.

February 9, 2018

Why Do I Write Code?

Today, I was read an article asking why you do whatever it is you do. If you are trying to sell to a customer, they’re not interested in what you sell but rather why you sell it. You may sell widgets, but people don’t buy widgets for the sake of having widgets – they buy widgets for whatever it is they do – the why. In my business, people don’t buy code – they don’t care about code. What they do care about is the why. Why do I write code? I write code to simplify business processes, to improve operational efficiency, to increase revenue, to ensure compliance with regulatory guidelines, and to simply make my client’s life easier. People buy those things! As a business, wouldn’t you pay for operational efficiency, increased revenue, and easier compliance management? Software can have an impact on every facet of a business and take your business to the next level. Don’t waste your time doing tasks that aren’t generating maximum revenue for your organization – find software solutions to solve those problems so you can spend your time having a real impact on your business by focusing your talents where they will generate the most revenue!

February 8, 2018

Finding Tech Jobs & Candidates

Scrolling through LinkedIn, I see people talking about how they can’t find jobs. Talking to companies, I hear how they can’t find qualified candidates. This seems to be the story all across the nation regarding IT positions. How can this be? Why are businesses unable to find candidates while there are candidates out there looking for jobs? As a software engineer and small business owner, I often look at job boards to see what companies are looking for, what trends are hot (based on something more than just twitter hype), and what direction the market is headed. Over the years, a few things have become apparent to me. First, software companies are constantly looking for employees that meet an absurd number of qualifications. Their requirements start to look like the checklist for a set of collectable cards. Is it really necessary for the candidate to be an expert in 15 different technologies? It’s particularly difficult now because of the sheer volume of technologies available for use. Languages, web servers, front-end toolkits, build environments, continuous integration environments, libraries, project management methodologies, and the list goes on. And under each of those categories, dozens of choices – C, C#, C++, Java, Scala, Python, PHP, etc. Eventually, job posts end up with so many requirements that you are literally looking for a needle in a haystack during your candidate search. Employers: Do you really need an expert in all those different technologies? I doubt it – you need a solid programmer who can learn the technology stack you have in place. The second thing I’ve observed over the years is that few developers actually have a drive to learn new technologies. How is it we still have programmers and other tech professionals that don’t know Linux? How can you be a software engineer and not have some knowledge of JavaScript? How many shops are using toolkits and frameworks that died years ago? Delphi is still out there, but would anybody actually chose to start a new project with this antiquated technology? Too many developers are stuck in their world – “I am a Java programmer, I use Eclipse on Windows, and I know Maven”. On both sides of the equation – companies and job candidates – there exists a real lack of desire to put in any meaningful effort. Companies could invest in their new employees and teach them the frameworks they need them to know. Just imagine the long-term benefit of showing employees that you care enough about them professionally to invest in them! Programmers could invest in themselves and learn new skills throughout their career instead of being content to use long-dead tech. Of course, neither side wants to invest any effort. So, companies end up without qualified candidates and programmers never go outside their comfort zone to learn new skills.