Assuring the Quality of the Enterprise Software Cloud

As cloud adoption accelerates, most organizations now pursue multi-cloud strategies. These span SaaS, PaaS, and IaaS offerings. This diversified approach prevents vendor lock-in, but also introduces additional test complexity. Quality assurance must validate functionality, security, compliance and resiliency, but this requires testing expertise across a matrix of layers and ownership boundaries.

The core challenge stems from the shared responsibility model underpinning cloud delivery. Customers and providers divide management, configuration and testing responsibilities differently across different parts of the technology stack. Where these boundaries intersect, gaps can occur and expose vulnerabilities unless consciously mitigated through expanded test coverage.

Who owns Cloud Software Quality?

A “shared responsibility model” for cloud security, provided by the UK’s National Cyber Security Centre and reused under the Open Government Licence (OGL) v3.0.

The scope of what your organisation can (and must) test is dictated by the distribution of control within each cloud delivery tier:

SaaS Cloud Quality

Assuring SaaS quality constitutes the narrowest test ownership for cloud customers, as providers manage the entirety of the infrastructure and platforms. Nonetheless, you must validate an awful lot:

• Authentication and access controls;

• Workflow and business logic mapping;

• User scenarios, rendering, and experience;

• Security of any confidential data shared.

PaaS Cloud Quality

In PaaS, clients control deployed applications while providers manage lower environment layers. This adds even more to your cloud quality responsibilities:

• API security profiling;

• Performance & capacity testing;

• Negative path & exception handling;

• Dependency version changes;

• Security scanning of application images.

IaaS Cloud Quality

With infrastructure under client control, in IaaS introduces significantly heavier validation lifts:

• Infrastructure hardening & baseline configuration;

• Network segmentation, routing and ingress testing;

• Workload isolation across virtual systems;

• API abuse & privilege escalation attempts;

• Resiliency against infrastructure component failures.

Accountability Gaps and Quality Risks Within Shared Cloud Ownership

While well-understood internally, shared responsibility demands broader quality assurance skills when engaging third-party capabilities.

As responsibilities subdivide across cloud delivery tiers, gaps readily emerge, allowing defects to be missed:

• Providers often validate just infrastructure/platform functionality, not usages ultimately running on top of them.

• Clients lack lower environment access to test security controls like firewall rules or to generate loads mirroring production.

• Blurry hand-off boundaries emerge around patching, logging, and monitoring ownership.

• Lagging documentation of configuration plus environment sprawl obfuscates change detection.

Further exacerbating matters, traditional testing groups frequently split along technology layers. App test, sec test, perf test may not integrate efforts into addressing shared quality risks. This encourages assumptions that a partner validated a specific area leading to blind spots.

With each side lacking full-stack visibility, accountability gaps emerge. Both sides overestimate the other's quality assurance investment. Providers stress robust platforms, assuming clients perform app security hardening. Clients expect providers to cover patching, DDoS and malware vectors as part of advertised security services, leaving gaps in practice.

Unifying Visibility and Quality Across Cloud Service Owners

Assuring quality across multi-cloud security and compliance obligations requires greater visibility and thorough testing processes, including instrumentation and observation across the entire delivery chain. This spans customer, providers, and integrators.

Solutions that model expected behaviours then generate proportionate test cases, offering an ideal mechanism for managing complexity. Models that accurately and dynamically capture systems end-to-end logic furthermore establish core artifacts for driving shared understanding across cloud delivery partnerships.

Some leading practices include:

• Maintaining architecture diagrams which map components, communication paths and trust/boundary lines between provider/customer-managed elements.

• Modelling identity and access workflows across cloud layers to auto-generate targeted authorization test cases.

• Cataloguing assets, configurations and services across cloud accounts/regions into CMDBs, enabling better test data targeting.

• Using chaos engineering techniques to simulate infrastructure failures or instances behaving unexpectedly in production.

Crucially, test data and use cases should not be limited to expected, happy paths. Cloud testing must stretch behaviours to address risks emerging from consumption growth, increasing user diversity and integration entropy over time.

While intricate, Quality Assurance approaches leveraging models, automation and shared visual artifacts can establish transparency and collective protection far exceeding isolated efforts. Unified understanding of customer and provider testing commitments allows accurately targeting residual risk.

For cloud consumption models to fulfil their convenience promise without compromising quality or security, testing requires equal innovation. Shifting left to prevent defects via early modelling collaboration combined with transparent observability into runtime environments together deliver the robust cloud outcomes that customers and providers jointly desire.

To learn how Curiosity can help you drive quality across your cloud development, speak to one of our quality experts today!