A Tester's Story of Adapting to the New World of Pipelines

As a software tester, how do you adapt when a new technology or process is introduced within your team? Do you panic? Do you accept it as a challenge? Do you research to find a solution along with the team? Do you reach out to people in the community to see how they approached it? Or do you just expect it to magically work for you without any effort on your part?

I was in similar situation when our team decided to adopt Continuous Integration (CI), as we were transitioning to a DevOps culture. I would like to share my experience of how, as a tester, it was to go through the journey.
 

It had been a while since we started to work in agile from waterfall. We took baby steps and evolved slowly. We were all happy, comfortable, and settled into the way we were approaching agile with our scrum process. Just when we were happy that we went through the agile transition from waterfall, now it was a new challenge of transitioning to DevOps. We had to embrace another new change of process, and it was the introduction to the world of pipelines - it was the CI/CD world!
 

As a tester you always like challenges, and this was challenging because I had to understand a new way of how testing fits in continuous integration, and embrace the new type of value it could provide while working on it.

As a tester I was very curious to know: 

• What it brings to my world of testing

• How would it affect current test  process

• What do I have to change

• What benefits do the pipelines get?

• What do they look like?

• How do they work?

And on and on. It was pretty confusing and scary at the beginning to visualize the change.

Why is our first instant emotion towards any change ‘Scary’? Why can't it be an excited feeling or enthusiastic feeling for trying something new - I asked myself this question. Then I thought to myself, it's as simple as this - When you don't know about something, you are always scared about the change, but if you feel safe and see it as a new challenge, you will definitely get excited.

Then I started to explore and read more about pipelines. It wasn't surprising that the more I was reading, the more I was confused. Then I decided to draw those pipelines and visualize them.

I sat with my colleagues and said: "Let's please visualize the pipeline by drawing it on a big white board." They  agreed and we had a whole discussion around these pipelines. And we finally agreed on this and it looked something like this initially:
 

That’s where it started to make more sense to me, and I got a better understanding about what exactly this continuous integration is - but all in theory though. What I understood at this point was that it is simply nothing but a step by step process to release the code once there’s a commit. Continuous Integration is a practice for integrating the code several times to receive a quick feedback without waiting long. Whenever the developer finishes development task and pushes the changes to the repository, that’s where the pipeline stage starts. Because it has stages, it is called as a pipeline I thought to myself .  

Now, a few more questions came to my mind:

• What are those stages?

• Who defines those stages?

• Who defines what should go in the pipeline and when?

• Will I have any place as a tester to suggest something?

• The biggest question was - will there be no manual testing at all?

We as a team had discussions and came up with the our own version of the pipeline because there’s no definitive structure for a Continuous Integration (CI) pipeline. We decided we shall run static code analysis, unit tests and integration tests as our initial stages of our pipeline. This stage would create a unique artifact which could be deployed on a test environment to run additional tests. As my understanding got better about pipelines, I started to suggest that we add more valuable tests to the pipeline.

Initially we added smoke tests to the pipeline which would run alongside the existing tests. The reason why I suggested to add smoke tests as part of the pipeline was to make sure the build is testable before it is deployed on a test environment for further testing. I just called it as smoke test, but we could call it a Build Verification Test or Build Acceptance Test too. If the test didn't pass, the build would be rejected and a new build would be released once the failed tests were fixed.

I could see each stage of tests running one after the other. It would stop going further if there was a failure at any of the tests or stages. Isn't that magical!!!

It was just the beginning for us and especially for me. I continued to understand and observe more and more, and then I discussed with my team if we could add regression testing to the continuous integration pipeline. But initially we did not wanted to add any long running tests to the pipeline as we thought the whole point of having these pipelines was for quick feedback. So we decided to run the regression tests as nightly run task instead of running it with every commit. But we had the goal to run these regression tests with every commit once our pipeline was mature enough, when we would decide to run those tests in parallel. But that was a long term goal.

Each morning I could see the results of which tests where passed and which were failed and the difference was that I could add the build number while raising the issue, so it got easier for a developer to find the root cause. It's not to say that it tells you where exactly where the issue is but they could easily check the recent code related to that build to see what went wrong. Instead of searching a needle in a haystack, it's like you know which part of haystack you need to look to find the needle.

 
I got to spent most of my time as tester defining the test strategy, exploratory testing, being part of the 3 amigos to refine user stories and acceptance criteria, checking the test feedback from the pipeline, and adding more valuable tests to regression tests.

As we got along and had more clarity and saw the benefits for having these pipelines in place, we decided to discuss a long term goal, where I suggested to have visual testing as part of the pipeline as well, from a testing point of view.


This was my actual pipeline journey into the world of testing! It was very challenging and interesting!!

Thanks for reading my story!!  Happy Testing!!!!