Leading the technology function can be challenging for many CEO’s. Yet, for other CEO’s it comes more naturally. Every CEO can learn how to become an effective leader of their technical teams by improving their leadership skills. There are things that will make a difference when leading a technical team, either positive or negative. In this four-part series I will cover those things that will help CEOs better lead the technical function in their company and avoid those things that undermine the effectiveness of their leadership.
In part one and two of this four-part series I discussed the foundational communicational strategies for a CEO leading technical teams. In part three I outlined what information a CEO should expect from her CTO to achieve engineering process transparency and how to get that information. In this final part, part four, I describe foundational technical concepts CEOs need to understand for effective leadership.
When it comes to the topic of software architecture it is important for a CEO to understand the difference between monolithic and service architectures, the latter being the preferred architecture in most cases. Simply, a monolith is an application where all the code is in one code repository and all the code in that repository must be deployed together. So, if the repository has 500,000 lines of code and a developer changes one line of code, the entire 500,000-line monolith must be re-compiled or re-built and then deployed into production.
As you might guess, this process can be complicated due to the size of the code base and time consuming because changing only one line of code means the entire code application must be regression tested and re-deployed. On the other hand, if the monolith has been broken up into 10 services, each with 50,000 lines of code, a one-line change only requires testing of a single service. As you may guess, the deployment of that service is much less complicated, and the testing of that service is much faster.
Most all applications store data. Therefore, CEOs need to have a foundation of how this is done. Typically, applications store data in a flat file or they use a database service. Applications create flat files by calling an operating system, such as Linux, to make and store the file. This is an inexpensive and simple solution. On the other hand, a database service can be used in combination with a query language, such as SQL. Databases are more expensive than using a file system to store data, but they also have many benefits.
When an application is deployed into production, there are typically three ways of doing it: As a disk image, as a container and serverless. If only one version of an application is running in production, it is called a single point of failure (SPOF). Most applications should NOT be deployed as a SPOF.
Product Development Lifecycle
There is a lot of confusion about the difference between a Product Owner and a Product Manager. Product Owner is a role within scrum, a process used by engineering teams to build software applications. A product manager is a title typically given to the Product Owner or the manager of those who work as Product Owners.
CEOs should avoid assigning features to an engineering team. Clever ideas are a dime a dozen! And ideas DO NOT translate into a useful product experience without tons of work by the development team. That work requires continuous guidance from the product owner. If CEO’s set feature direction and are then unable to provide daily stewardship over that feature development to the product engineering team, the feature vision typically veers off course quickly. Therefore, those that provide feature vision must also be involved with the software development process daily. Instead, CEOs give business goals or objectives to a product development team.
Done is a technical term in software development. It’s when a developer gets to put points up on the scoreboard for her team. Literally! Stories are a description of developer work or tasks. The complexity of the story is established in story points. When a story is done, the team gets to count the points. The points are a unit of accomplishment and a measure of the team’s velocity. CEOs should know when their developers consider a story to be done. Ideally, it is when the story has been released to the users who will be using the feature.
Velocity is a measure of the team’s speed in story points. It is super important for a CEO to understand development team velocity. It is analogous to the current of an electrical circuit, or the horsepower of a car engine. The CEO’s job is to distribute the engineering power to the goals and objectives she has given to the team. She cannot do that without knowing how much power is there.
A code repository is where software engineers store their code. A code repository has a main branch. The main branch is where the current version of the software is stored. When developers want to change some code, they create a branch off the main branch, for example, a feature branch or a bug fix branch. When the developer is finished with her change, they make a pull request that pulls their code into the main branch. The pull request should include automated unit and integration tests. A unit test tests a small part of the code. Integration tests run all the unit tests that have been written and validate the code. Once the tests are passed, the new code is merged back into the main branch. Once merged in to main, the code is released in production. This is when the developer can report that her work is “done.”
This is the core engineering process. Efficient teams run through this entire process every day and multiple times a day. Inefficient teams are not able to release software this quickly and in some cases the process can take weeks. For healthy development teams, this process will take no more than 4 hours.
Software applications run on virtual machines. A virtual machine is a computer made fully out of software. Virtual machines run on physical hardware machines. Many virtual machines can run on the same physical hardware machine.
Containers are smaller versions of a virtual machine. They are optimized to run code more efficiently. A physical machine running containerized applications can run more applications than one without containers. Therefore, containerization has become popular; containers get you better “mileage” out of the physical engine.
Orchestration is the management of the containers that are running your application code. A monolith is too big typically to run in a container and harder to orchestrate. A monolith should be decomposed into smaller services that can be containerized and orchestrated. This gives you the best possible gas milage for your buck!
When something goes wrong with the release of new software, the team should roll back the change. This can be done efficiently through orchestration. With smaller services, this is easier and faster to do, minimizing disruption to the customers using the software. For monolithic applications this will take longer to do and result in longer downtime.
When something goes down, an alerting system will kick in letting the operations team know there is a problem. Alerting relies on monitoring. Therefore, you must have a good monitoring system. This takes engineering effort/velocity to build. Be sure to direct velocity to build an excellent monitoring system.
The monitoring system writes errors to a log. Logging is super important to stable software services. Be sure you have an excellent logging system. Excellent logging collects information from all the monitoring systems and stores it in one location. This is important because teams must have a clear, 360-degree view of the platform performance. Without a central log, teams are blind to what is happening. Be sure to have an excellent logging system.
When something bad happens, it is called an incident. The root causes of that incident are called problems. A single problem can create many incidents. Incidents have a severity level, typically S1, S2, S3, etc. S1 is the highest severity level and usually means the company’s revenue will be impacted. It is critically important that you have defined criteria for triaging incidents into their proper severity levels otherwise valuable engineering velocity may be leaking out to work on low priority incidents. This happens all the time and is a fundamental problem with most engineering teams. The typical symptom a CEO will see resulting from a velocity leak due to lack of an efficient triage process is long wait times. The CEO must wait too long for things to get done. If you are experiencing this look first at where engineering time is going and in particular look at incident interruptions.
Over time, the number of incidents should be going down. Also, the number of problems should be shrinking. Make sure you have a KPI to track these with your CTO. They are a key measure of success.
One of the most common reasons for poor engineering velocity is due to teams having to wait for something another team needs to complete for them. Slow teams typically have too much dependency on others and must wait too much.
Cross functional teams are the best way to reduce the dependency that slows velocity. By making teams independent (not reliant on other teams or persons), they can get stuff done and control their outcomes. This is by far the most desirable state: Independent, cross functional development teams.
Engineering departments based on functional orientation (front-end development, back-end development, etc.) have too many dependencies and are therefore inefficient. Avoid large functional departments and managers that run those departments in your engineering organization. Replace them with cross functional teams run by managers focused on delivering a business outcome.
Corporate security is the responsibility of your CISO. However, when it comes to the security of the software that you build and sell as a product or service, or that you build as part of your tech enabled business, the security of those applications belongs to the CTO. This is a particularly important distinction, and it is not obvious to many CEOs. Often, the CISO is placed in charge of all security. For software product companies and tech enabled companies this is not ideal because it creates a moral hazard for CISOs, and it creates a dependency for the CTO that slows engineering velocity.
Also, when your company builds software or is a technology enabled business that relies on software it builds, security is an attribute of the architecture of that software. It is the same for other non-functional attributes such as reliability, scalability, maintainability, etc. As a CEO, you’d never consider moving accountability for reliability, scalability, or maintainability away from the CTO. The same applies to security.
Encryption is used to protect both stored data and data moving across the network. Not all data needs to be encrypted. Encrypted data takes up more storage and requires more power and time to decrypt. It is inefficient to encrypt all data. Therefore, data classification is needed. You should be aware of your company's classifications and whether or not the data is appropriately protected vis-a-vis the classification.
Authentication and authorization are two services used to protect data and the company assets. It is important that both are disabled immediately for employees who have left the company. It is also important that authorization only be granted to those that must have access to secure data. Review the authorization schedule with your CISO on a regular basis. Also, review the process whereby accounts are removed when someone leaves the company.
Finally, scans, intrusion prevention (IPS) and intrusion detection (IDS) are important tools used by your CISO to protect secure data. It is important to understand what each of these do and why they are needed. This analogy may help. Intrusion prevention is analogous to a wall you put around a secure building. It prevents outsiders from entering. You need to understand the metaphorical height and strength of the wall. For example, a picket fence will keep out most law-abiding citizens but not a thief.
It is quite easy to overspend and overcomplicate data access unnecessarily creating bad user experiences and frustration. Collaborating on decisions with your CISO will lead to much better outcomes. Avoid delegating this under the belief that it is too technical or something the CISO should just handle. Neither are correct.
Intrusion Detection (IDS) helps you find unauthorized activity in your network and servers. It is a second layer of protection if the IPS fails. Both IPS and IDS are not fool proof. There are bugs in operating systems and other software and hardware that allow bad people to break in. Vendors release patches to fix this, but only if they know about it. If they don’t, bad people can get in.
Finally, scans are used to find weaknesses in your security. It is a good idea for you to review the scans quarterly to be sure that adequate resources are put into addressing weaknesses in security and to be sure that nothing is getting pushed under the rug.