The following are scenarios that demonstrate some core recommendations when organizations are performing Knowledge Management, Continuous Integration, Delivery and/or Deployment.
Artifact Sharing/Knowledge Management
How does one capture the knowledge of a team that is continuously iterating and developing solutions? All the artifacts and notes created for each capability tell a story of who, when, what and why. Using these tools we can go back and forward in sprints to learn, educate and remember.
- Agile tools such as Rally and JIRA capture document the user stories, participants, team communications, releases etc.
- Test Automation tools like Ready!API capture the API invocation flows via TestSuites and TestCases
- SwaggerHub and Swagger provide a means to define, version and communicate APIs to team members, partners and consumers
- Using the Swagger definition, one can incorporate the meta-data into Ready!API solutions, Amazon Web Services, Microsoft Azure, WSO2 solutions and other third party products
- Source Code Management solutions keep track of all versions / submissions and associate comments
API Virtualization
When developing in an iterative manner, changes to backend systems will require days/weeks to complete. This makes it very challenging for teams that are dependent on APIs to perform their work including Front End Developers, DevOps / Test Developers and other Third Party Developers.
A common approach which helps lessen the burden allow for parallel development is the use of Mock services or API Virtualization. Consider using the following solutions to increase your team’s velocity:
- Smartbear ServiceV to simulate to be built endpoints / API logic
- SwaggerHub generated Servers to expose a public simulated endpoint / API logic
Test Automation within Continuous Integration
A key component in Continuous Integration is the ability to verify the software being integrated provides both the new capability(s) and does not introduce regression in the existing capabilities. Test automation provides this capability at the User Interface, API and Data Store level.
- Have developers create Unit tests to be focus on new components
- Have developers create TestSuites / Testcases to exercise the API functionality and use Assertions to confirm the Transport Codes, Responses and Headers match expected behavior in both the successful scenarios and failure scenarios
- Have DevOps create Data Driven Testsuites / Testcases to flood the system with more realistic information to isolate / identify data related issues
- Have DevOps execute Security scans to pinpoint unexpected behavior due to malicious consumers
Test Automation within Continuous Delivery/Continuous Deployment
Where continuous integration focuses on the integration of new features into the existing code base, continuous delivery and continuous deployment move the capability up to production or into production via automation. In order to control the movement of the capability from one environment to another, gating needs to be performed. Automating testing provides a means to introduce the gating logic.
Tools such as Smartbear’s Ready!API provide a rich set of capabilities to perform functional, data driven and security testcases. This, coupled with complex assertion support, allows teams to accurately define complex logic to gate Continuous Delivery or Continuous Deployment pipelines.
- Have DevOps leverage Smartbear Ready!API TestSuites/TestCases that are specific to the environment being validated. In the QA environment, Data Driven tests with appropriate assertions could be implemented.
- In another environment, Integration Testing may be relevant and thus require using Smartbear Ready!API to support invoking a series of APIs in support of a business process. Ready!API provides a simple way to define numerous APIs, SOAP Services, Databases and JMS endpoints that can be used within a Testcase / Testsuite to accurately reflect the real process.
System Testing of Automated Infrastructure
Organizations that have leveraged automated infrastructure in cloud providers such as Amazon Web Services, Microsoft Azure or Google Compute test to make sure their infrastructure configuration supports resiliency. Netflix has led the effort in developing open source tooling to kill / disable infrastructure (e.g. Chaos Monkey). These outages will often test features such as:
- Container self-healing
- Auto-scaling groups
- DNS Failover
- Elastic Load Balancing
- Multi-AZ failover
- Multi-Region failover
In order to verify applications that have been architected to fail, It is important to simulate production level traffic to exercise APIs and Applications with tools such as Smartbear LoadUI. LoadUI can generate accurate API requests and appropriate level of virtual users.
This in combination with Chaos Monkey can allow teams to simulate system level outages and deeper understanding of how infrastructure behaves. These behaviors provide valuable information to DevOps personnel who can then develop a playbook and valuable experiences.