Author: Mihai Avram | Date: 03/21/2021
Everyone in tech says that testing your code is important, but what does this mean? We are constantly bombarded with new testing methodologies and frameworks, and your project partner comes to you excited about involving some new Integration Testing procedure? What should you do when there are so many overwhelming options to make testing work. If you feel this way, this guide is for you, and is meant to explain the most important methods, frameworks, and approaches to testing no matter what your situation is.
What is Testing and Test Driven Development and Why is it Important?
Code testing and Test Driven Development (TDD) are merely just methods to ensure that the code you are writing is doing what it is supposed to do. This may not be needed when you write simple code that adds two numbers; however, as a codebase becomes more complex and deals with many pieces of code that interact together, there are a lot more areas where it could go wrong. This simple guide will introduce the various types of testing methods available out there, well-vetted testing frameworks for popular programming languages, and how to approach code testing from a practical perspective. Let’s begin!
Types of Testing
The main types of testing available out there are Unit Testing, Integration Testing, End-To-End Testing, and Acceptance Testing. Test Driven Development (TDD) is also a meta-concept that encompasses some of these testing methodologies so we will cover this, as well as variations of it. To make the definitions more applicable let’s also envision the following example.
Running Example – Suppose you have a code library that you have built for educators that takes in someone’s grade on a test, analyzes how good it is in comparison to other people’s grades, and returns a report on how they stack up against their peers.
Unit Testing
Informal Definition:
When we test one specific component or function from our codebase and make sure that
it is doing what it is supposed to.
Example:
Let’s say that in our code library we have a function that has to compare someone’s grade to other people’s grades. This comparison function should be tested in detail to make sure that it is doing what it is supposed to.
Integration Testing
Informal Definition:
When we combine two or more specific components that work alongside each other, and test that they are working together as intended.
Example:
In our grade benchmarking code we have a function that reads in someone’s grade and after that, we pass it to a function that compares that grade with other grades. These two functions can be tested together by passing in different inputs and data through them and making sure that the code does what it should and there are no bugs in this functional interaction.
End-To-End Testing
Informal Definition:
Testing a whole codebase and process from the beginning to the end, and all the parts or functions in between.
Example:
We test all of our grade benchmarking code from one end to another. This will entail providing it with the grade of an individual, and running that grade through the benchmarking. Then after that we call the final function to generate a report about the individual. This test covers the whole breadth of our codebase and of course is only applicable for functions and processes that are tied together in one larger process flow that includes an input and an end result.
Acceptance Testing
Informal Definition:
A testing method that really expands upon End-To-End testing and adds the extra requirement that the output of the process should abide by important constraints, business logic, or policy to make sure that it is what we expect all the time, and in the format/style we want.
Example:
Let’s imagine that the output report we generate for the user needs to always have the individual’s grade present, and the grades of other individuals, all rounded to the second decimal place. An acceptance test (among many) would test that these constraints are met and that all the grades present are all rounded to the correct decimal place and there are no null items.
Test Driven Development (TDD)
Informal Definition:
A testing practice and philosophy where one thinks of and writes test cases of a specific function or code task before the actual code is written. This forces the programmer to think of various ways the code can be faulty and then write code with that aspect in mind so that bugs are avoided in the future.
Example:
Let’s rewind the clock and pretend that none of the grade benchmarking code has been yet written. We first have to start with the function that can take the grade as input. Knowing what this piece of code or function should do, we come up with various test cases such as “Is the grade input a positive number?” or “Is it less than the maximum possible grade?” etc. Only after we finish the test cases, we write the actual function and ensure that the function we write will pass our test cases.
Acceptance Test Driven Development (ATDD)
Informal Definition:
A type of Test Driven Development where many team members that bring different perspectives create tests. These different perspectives could cover a wider range of perspectives than just one individual’s TDD ideas.
Example:
Three team members need to code the part that involves the end report generation of users’ grades for the educator. Before they start writing any code they come up with various questions such as “What if the input was of this particular format” or “What problem do we need to solve and show in the report” etc. – which will come up with a variety of tests that can cover the possible solutions to these questions.
Behavior Driven Development (BDD)
Informal Definition:
Yet another adaptation of Test Driven Development and Acceptance Test Driven Development where users must cater their goals to outside business outcomes by asking many questions about what the function to be implemented needs to achieve.
Example:
Three team members and educators meet and discuss the functionality which involves the end report generation for the educator. Before they start writing any code they come up with various questions that are related to the business cases and what is expected in the end use-case specification for the final stakeholders (e.g. educators that need to view the grades for their students). This is an extra step from Behavior Driven Development by focusing more on the end users and stakeholders.
How to Write a Good Test Case
- Create thoughtful positive test cases to ensure that the most common and popular uses of data will work successfully. For instance, in a banking application, one should test that a currency of 1 can be charged, 10, 100, and even 1,000, and then for 10,000 it should call the user first before making a transaction of such a large size. Make sure these steps work and focus especially on tests that are as close to how your application functions in production as possible.
- Create thoughtful negative test cases by supplying non-conventional or unacceptable data and making sure that the code is not doing what it is not supposed to do. For instance, in a banking application, when supplying “1000” as a string instead of as a number, it should throw an error instead of charging that value to the user’s account.
- The more variations of positive/negative test cases, the better, however, focus on the ones closest to what would happen in a live environment for your project.
- Document your testing code very well with insightful comments and docstrings.
- Make sure that tests are idempotent, meaning that if they are executed many times, the runs are independent of each other. For instance, let’s imagine that some set-up function is called that initializes a bank account. We want to make sure to remove that bank account object before the next step so that it does not interfere with the next test. In short, make the set-up and tear-down functions for test robust, and transparent.
- Keep unit test cases separate from each other and don’t blend them together except for when running integration and end-to-end tests.
- Involve more testers, and especially stakeholders that are closer to the experience of the end-goal (e.g. the bank manager in a banking application) and embrace Behavior Driven Development as much as possible.
- Make sure that the names of the test cases are reflective and transparent of what the test cases are doing.
- Start with unit tests, then build up to integration tests, and finally end-to-end tests.
- If a test case is more manual and administered by a user:
- Create detailed and easily interpretable test instructions
- Provide any test data if needed
- If there are preconditions, make sure they are well documented (e.g. highlight that the user must be on the profile page for the test to work)
- Have expected results present for the tester to reference
- If the functionality and results are complex, document these items so that the tester can better interpret the results and be able to test them successfully
For more manual test case scenarios where test cases are administered mostly by quality assurance (QA) specialists and not automated tests, the following video explains the scenario very well.
Popular Testing Frameworks for Popular Programming Languages
Python – PyTest is the most popular framework and best for unit tests; however, if you have different needs on the project, such as a framework that includes behavior driven development, then Behave might do the trick for you. Here are some great articles explaining some of the best Python frameworks for different use cases.
Top 6 BEST Python Testing Frameworks
Top 8 Python Testing Frameworks in 2021
JavaScript – Jest is a test framework developed at Facebook and the most popular framework for JavaScript and React. It works with other frameworks like Vue and Angular as well. If you have other preferences such as a full-stack testing framework for Node.js, Mocha has you covered. Here are some articles to point you in the right direction.
Best JavaScript Testing Framework? 5 to Consider
Top 5 Javascript Testing Frameworks
Java – JUnit a popular testing framework that integrates with all major IDEs. There are many tools, however, so pick one that is most aligned with your use cases from the following articles.
Top 10 Testing Frameworks and Libraries for Java Developers
Test Automation – Selenium is one of the best frameworks for automating test cases on the web, and if you have more specific test automation use cases you can check out the following article that explains which libraries are best for which use cases.
Top 10 Test Automation Frameworks in 2020
If You Have an Unlimited Budget What You Should Do?
If you have a lot of resources and a big team, it is highly encouraged to prioritize testing and test cases. A general rule of thumb is to capture 70-80% of code coverage which is essentially the percentage of code that is covered by test cases. This percentage should also increase with the cost of failure. For instance, if we are building a rocket to get us to Mars, a failure may have a catastrophic result with many lives lost, not to mention all the resources and time that went into building the rocket to get us there. So code coverage in a critical case like this should be 100%. The same goes for code that has human lives in its hands such as autonomous vehicles.
What Should You Do at the Very Least, No Matter What Your Situation is?
If you don’t have many resources, time, and no team to build test cases with, it is still good to have some test cases present. Which test cases though? Focus mostly on end-to-end tests and tests which touch the most critical parts of your application. Let’s say you are building software that is in charge of delivering Personal Protective Equipment (PPE) to healthcare workers, then make sure that any piece of code or algorithm in charge of the delivery process is running under test cases. All the other more peripheral and non-critical functions can wait until you have more time, funds, or a larger team.
Conclusion
There you have it, I hope you understand the current testing landscape a bit better and can create more robust and error-free code for your projects. There are many test cases and theoretical frameworks for all the programming languages, so if you are using a more niche framework such as Rust or Ruby – a google search will be your best friend.