Nowadays programmers are hardly working alone, most of the cases they are developing as a part of a team. Sometimes this team has just 3-4 members, but there are several huge projects with over 1000 developers. To organize their work is not easy. Not only from management point of view, but also technically.

First of all it is needed to track the different versions of the software. For that purpose there are several different version control systems (like Git, SVN, Mercurial etc.), but most of the cases it is not enough to have a version control system, you need to also make sure that the different versions are correct.

What is software integration?

Software integrator is a dedicated role at several companies. What does it exactly mean? Different components of the code and different part of the components are done by different team and different developers. They are all doing their part and sometimes doesn’t have an overview on the whole software. So in the end they are all done and they realize that even if their part is doing what is expected their code can not work together. Sometimes even can not build. It is because of different interfaces, different expectations from interfaces etc. Or simply after integration the software is becoming too slow or it is using to much memory. There are also different standards (coding guidelines etc.) which are not followed correctly in all code. These tasks are all the responsibility of the software integrator to make sure each part of the code is fulfilling the requirements individually and then merge them into one codebase and make it work. This work sound maybe easy, but most of the cases it is challenging. Furthermore it takes a lot of time. Just imagine, you are getting the work of several weeks from 10 developers and you need to make them work together...it’s a nightmare, right? Luckily there’s a strategy which make it easy, it is called continuous integration.

What is continuous integration?

The idea is continuous integration is not to wait until the chaos is becoming too large, but integrate the software as soon and as often as possible. Every day, or even after each change. Of course manually it is not possible, so the idea is to automate as much as you can. First of all you need a version control system, which is following the actual status of the code.

There are several checks which you can run automatically on your code base:

• Build the whole code
  
  It is especially useful if you are working on multiple compilation targets, so you can build it for all of them
• Run unit test
  As already discussed in my post about Test Driven Development it is also good to cover your code with unit tests, so that you make sure that the others who are changing your code are not breaking the already working functionality
• Run higher level tests
  You can create also automatized component or integration tests and you can run them any time to check if the already implemented functionality is still working
• Run static code analyzer
  There are several tools which can run a static check in your code. It can detect potential bugs (like usage of uninitialized variables etc.) and it can also check for the expected coding style or give suggestions for more effective code
• Run dynamic checks
  There are several tools which can detect typical missusages like memory leaking or hanging of the program
• Do measurements
  You can always measure the runtime of critical scenarios or the memory usage of your code. You can even measure the code coverage
• Run additional scripts up to your needs
  It is always project dependent. For example in one of my projects we had a script which collected all TODO comments from the code, so that we could make sure before the release that all of them are already resolved.

You can run all these checks in any state of the code and give feedback immediately to the developer team though a status monitor tool or by e-mail notifications. So that the team has always an idea about the current state of the code base. It is a good strategy to keep all tests green.

Scale your system

To run all these checks on your code can take a really long time if you are working with a huge code base, especially because compilers and static code analyzers are usually extra slow. So it makes sense to run everything all the time, because it would cause a too long feedback loop. If you are getting the notification first after one day that your code broke something that does not help a lot. So you need to scale your integration framework.

A set of checks needs to be run after each change of the code base (after each commit), like compilation and unit test.

You can also set up a nightly job which is running several scripts every night on the code, so that the developers are getting feedback every morning: compilation, unit tests, higher level check, some part of the static analyzers.

Plus you can set up a job which is running all the checks, but this one you are letting run only for example once a week or before a software release.

Use frameworks

You could implement it all yourself with some scripts, but fortunately there are several frameworks which are already supporting this kind of functionality.

One of the most popular is Jenkins. First of all it is free. It is also very flexible. You can set up several kinds of jobs with several trigger actions. You can setup also different kinds of notifications. It has a plugin for several external tool (GIT etc.), but you can integrate any tool with a short script on any scripting language (python, bash etc.). It is created nice reports and diagrams which you can just show to your management.

What I want to say here is that you can setup such a system easily and fast and it makes a lot of benefits.

Define your integration strategy

You need to also design your integration strategy. If you are integration every change immediately to the final software then what can you do if it is breaking the build? It needs to be fixed and in the meantime everyone is blocked. That’s why it is a better idea to run first the checkers and integrate it to the main version only if all checks based.

There are also processes where you are developing the different components independently and you are only merging some dedicated “release” versions of the to the main software.

And there are much more strategies. You need to decide what are you merging to the main software when and how. Most of the strategies has its own advantages and disadvantages. You need to define which one is fitting the most to your project.

How to do a proper technical training for developers

Most developers really appreciate to have a chance to take part on technical trainings paid and organized by their company. It is really a good opportunity to learn something new and make the daily work more effective. There are professional companies offering trainings, but at several companies there’s an internal training system, where trainings are held by the employees of the company. Unfortunately a part of these trainings is just a waste of time and money, because they are not transferring the knowledge in the right way. To do a proper training is not easy at all, next to the outstanding technical knowledge very good presentation and teaching skills are also needed. In the post I collected several points based on my experiences which you need to act if you are holding a training for developers. And I think most of these points are valid for any kind of training.

1. Know the knowledge level of your audience
   
   

   Before the training you should understand what is the current level of the group. You can have some personal talks with them or you can send around a short questionnaire some days before the training. You can also do a short introduction round at the beginning where everyone can tell what is their experience. Adapt your training based on this information.

   
   
2. Set up an agenda, but also a bit flexible with that
   It is necessary to have an agenda for the training where you are collecting which topics need to be addressed. But also a bit flexible, if one topic requires a bit more time don’t change to the next topic without really finishing it.
3. Less is often more Don’t try to address all topics. Try to collect the most important ones and let your audience understand that topics better. Addressing many topics usually means that the audience does not understand any of them properly.
4. Plan breaks
   Noone can pay really attention on you longer than 90 minutes without brake. After 90 minutes at least 15 minutes long break it needed.
5. Don’t plan too much for one day
   It is not effective to plan too much for one day. Sometimes I attended trainings which were 10 hours long. I don’t remember anything from the last 3 hours. The training shouldn't be longer than 8 hours per day (including breaks).
6. Teach way of usage instead of pure facts
   It makes much more sense to show some working examples instead of showing slides with a lot of data, especially in the IT world. Everyone can find the raw data on the internet. A training should be more practical
7. Be interactive
   Always try to communicate with your audience, ask them about their experiences, ask if they understood everything clearly, check if their really understood. You can also do some team work session.
8. Do practical exercises in teamwork
   If you are doing practical exercises (like implementing a small program) do it always in teamwork. So that the group members will talk to each other about the material and they will point out themselves what they don’t understand clearly or if they misunderstood something.
9. Concentrate on practical usage in their situation
   Always try to find examples from the field of your audience. So if you are doing the training for people working in banking sector don’t bring you examples from automotive business.
10. Be open for questions
    Always ask it your audience have questions and give a full answer. Never tell that “sorry I don’t have time to answer this question”.
11. Check what their understood from the training
    It is not enough to broadcast the information. You need to make sure if your audience really understood what you are talking about. If not just try to explain it from another perspective as well.
12. Be available after the training
    
    A training does not finish at the end of the day. Your audience retrieved a lot of information, they need to process it, to try it out in practice. Most likely they will have questions after some days of weeks. Be available for such questions, either online or schedule a follow-up meeting.

Different aspects of Test Driven Development

Introduction

“Testing before the software is done? What a silliness? It is totally crazy. It goes against the theory of testing. These guys should learn first how to develop software.” I heard these sentences from a software architect who had around 20 years experience in the software  business and I think he was a really good professional, he was just a bit conservative. That time I did not have enough experience to have my own opinion about the topic. Now I already have.

The Test Driven Development is one of the buzzwords in the world of programming which became popular during the last years. Each fancy company is advertising itself by being agile and doing TDD. But what does it exactly mean? Does it really change everything what we learnt about software development?

The theory of TDD

Based on Wikipedia: “Test-driven development (TDD) is a software development process that relies on the repetition of a very short development cycle: requirements are turned into very specific test cases, then the software is improved to pass the new tests, only.”

Practically that means that is you are developing a peace of code you are creating the code and its unit test parallel. First you are implementing one unit test case, which is testing against the first requirement against your code. Then you are running your test case which will of course fail. Now you are implementing a peace of code which is making your test case green. Then you are implementing your next test case and extending your code in a way to pass the test. If you think that you could change your code to be cleaner just change (refactor) it and check if the tests are still green. Repeat the test until your code is not fulfilling all the requirements.

How I interpret TDD

Let’s see how do we develop by default? We are writing a piece of code. After a while we are trying to run it to check if it’s working. If it is a library or something that is difficult to just run we are often implementing a short demo code which is not doing anything else just invoking our functionality with some predefined parameters and we are using it to check if our code is correct. In other cases we are figuring out a certain scenario in our program which is invoking are functionality and we are doing this scenario every time we want to check if our code is working correctly. If our code is not doing what we are expecting then we are changing it and run it again. Once it is doing what we want we are extending it with several new lines (some new functionality) and checking if it is working. We are repeating these steps until your code is not doing exactly what we want to have.

Are you developing in the same way?

I think most of the developers are doing something very similar.

Is it similar to test driven development?

Yes, there is some similarity.

Which disadvantage do I see at this methodology?

• Run the “test scenario” often takes long
• You are always testing against the same input parameters
• You are always concentrating only on the last functionality you added
• If other developers change your code later, they won’t have any idea what should your code do

Sometimes it is also required to implement unit tests (due to project requirements or whatever). So that you are trying to implement some unit cases to an already existing code. You are implementing them in a way that they will pass for sure. Furthermore if you code is not structured properly it can be really a nightmare to cover it by unit tests.

How does it change in case of TDD?

It is basically nothing else then hard coding your “test scenario”. Instead of manual tests (running your scenario manually and checking yourself the results) you are using automatized unit tests.

So first you are figuring out what should your first piece of code do. You are implementing your first test scenario in a unit test. Then you are coding it and checking your test scenario. If it is not working, then you are fixing your code, otherwise you are extending your test scenario (by adding new test cases) and implementing it.

Sometimes it is also required to implement unit tests (due to project requirements or whatever). So that you are trying to implement some unit cases to an already existing code. You are implementing them in a way that they will pass for sure. Furthermore if you code is not structured properly it can be really a nightmare to cover it by unit tests

The only point is that in the classical way most of the cases first I’m implementing and then I start to think what my code should really do. In case of TDD first I need to define what should the next piece of code do. Here I’m handling it a bit flexible and sometimes I’m just implementing the code first and then adding the new test case. I know, it is against TDD, sorry.

By using this way of development:

• You can tell that you are doing TDD
• You can check if your code is doing what it should by one click in some short seconds
• You don’t need to repeat the same manual test scenario always
• You can easily set different input parameters to the all the corner cases
• You are always running and concentrating on all test scenarios, so you realize for sure if you broke some previously implemented functionality
• If anyone is changing your code in the future, they can check if they did not break the original functionality of your code
• Your code is for sure well-structured to be testable, which is one criteria of the good code quality
• You can anytime refactor your code without taking the risk that you are braking the functionality
• You have unit test with a good code coverage

The only disadvantage I can mention is that implementing good test cases takes time. But running manual scenarios 1000 times also takes time…

There’s no black magic, right? You don’t need to change everything in your current practices.

The relation of TDD and classical test methodologies

In the classical development processes (waterfall, V-model) there are requirements. The software development team is creating a software design from that and they are implementing it. After that a dedicated test teams checks if the software is fulfilling the requirements. It is really important that the developer and the tester should be two different person. So that if one is misunderstanding a requirement the other will realize it.

But the requirements are usually high level functional requirements, so they are not tested by unit tests, but by some higher level tests, like system tests, components tests or integration tests. So the tests done by the test team are on a higher level. The unit tests are usually done by the developer team based on the software design what was also done by the developer team. In 99% of the cases it just means to implement test cases which are passing on the already existing code.

TDD is replacing this kind of unit testing activity. But it doesn’t have anything to do with the higher level testing. I think such tests still makes sense before a software release, so good news: you don’t need to leave your classical testing process, it is not against TDD.

But let’s see further: what happened then at testing phase? The test team finds a bench of bugs, creates bug tickets and send them to the development team. The development team is fixing the bugs and creates a bugfix release. The test team if testing again and realize that some of the behaviour is still not correct, so a new bugfix release is needed. To improve this process the higher level testing can also be done (at least partially) by automated tests. And it is also a nice idea to implement these automated tests before the software development is done and provide them to the software team. So that the software team can run the test cases on its own and in case of failing test cases they can fix the code before releasing it. So that they can save a lot of time both for themselves and for the development team.

The very best that can happen is to run all test cases in the continuous integration system, but it is already a next topic (maybe for next time).

Summary

You can see that test driven development does not differ much from the usual way of development and it also does not hurt the main points of the classical testing approaches. It can take some time and requires practice to implement proper test cases, but it makes really a lot of benefits on long term for your project. So I can just propose to try out TDD, but you should always handle it flexible and don’t expect that it is solving all the problems of the software development.

How to plan a project?

Introduction

As I’m talking with other software developers I really often hear the points from them like: we are just suffering due to the bad planning of this project, for sure the company is winning a lot of money from our work, I don’t know what the project managers are doing all they long etc.

For getting bit better overview on this topic I would like to write a short introduction to project planning and management, to clarify some unclear points.

There are several different working models, project planning models and situation. Since I’m a developer I also don’t have deep knowledge, so I really would like to show only on high level how all these things are working.

Planning

Before starting with a project the project should be plant. It needs to be decided what is the exact goal of the project and what are the so called included activities, like: is hardware integration part of the project? Which level of testing is required? In the next the used technologies needs to be decided at least on high level: which programming language should be used, which other software and hardware will be needed during the development?

Based on its results a skill analysis should be done: which skills are required from the team who will work on the project and on which level? Like if it’s a project developed in C++ with QT framework for GUI, tested with Google Test framework, it needs to run on both Windows and Linux then the team will require experience with GUI applications, knowledge of software architecture of GUI based applications, strong C++ and QT knowledge and experience with GTest and GMock.

Once it is done it is time to go a bit deeper into the project. The whole project needs to be splitted into smaller steps, so called milestones needs to be defined, like:

1. Milestone: GUI prototype, without backend functionality

2. Milestone: User handling related functionalities: login, logout, registration of new users, change password

3. Milestone: Working X, Y and Z functions

…

N. Milestone: Application with full functionality, fully tested.

It also a good practice to plan an N+1. milestone for bug fixes. The different milestones should have more or less the equal size. And this practice is the same for classically managed projects (V-model etc.) and for agile projects.

Estimation

The next point is the estimation of size of milestones. This is still a so called high level estimation. To be able to do it it needs to be listed on a high level what all should be done for one milestone and a rough time estimation needs to be added. The estimation can be done in manmonth, manweek, working day or even in sprints or story points. This estimation is the basis for calculating the due dates of the milestones and the needed team size and setup in different periods of the project. And this all will be a big basis of cost estimation.

How to calculate the price

Let’s see what all needs to be included into the price? Usually the biggest cost is the salary of the developers. Usually it is not calculated based on the actual salary, but based on an average salary of the given knowledge level. Take the example: based on the estimations the project will take 10 months and for the whole length we will need 2 senior developers and 4 juniors. The average monthly gross salary of a senior developer is 6000 dollar and it is 4500 dollar for a junior developer. The the whole costs of salary for the development team is 300k dollar. It also needs to be considered that there are managers, who will also work on this project. And usually there is additional stuff at the company: HR department, high management, sales etc. who are not working directly on projects, but the salary needs to be covered by the project incomings. These costs are usually called overhead and setup in a certain percent of extra costs.

Additionally a training budget needs to be plant for the project. You need to also think about hardware and software license costs. Probably additional travel costs needs to be also considered if the customer is not in the same city as your company.

You need to also think about office rental fees for the team.

Once everything is collected and estimated (based on the estimated length of the project and size of the team) you need to consider that the company would like to make some additional money from the project and the fact that if things are not going well and additional costs are coming into the picture (ex.: the project will take longer) the project should be still beneficial. So you should setup a price for the project which is pretty clearly higher then the costs.

Tracking of the project

Based on all these estimations the project manager will make a time and a cost plan for the project and he will keep on track the current project state. So if the team starts to be behind the estimated time the project manager needs to react to be able to keep the project beneficial for the company.

In spite of this there are still a lot of projects which are earning less money than the costs. In such cases the company needs to move budget from other projects.

Sometimes companies are starting projects where it is pretty clear right at the beginning the it won’t be beneficial. It has usually a strategic cause: they would like to win a new customer or build up knowledge base on a specific field.

If you need to plan the whole project in advance that is pretty risky. Luckily there are other approaches: the times based one, when the customer is paying after the real working hours done on the project. In this case there’s almost no risk, but of course the potential benefits are also lower. As an alternative you can use the agile approach when each smaller tasks has it’s on price and you can regularly decide together with the customer which tasks should be done and for which price.