Testing and Quality Assurance

Introduction

Software testing is a core practice for ensuring software quality. This article covers the testing pyramid, TDD, testing tools, code review, static analysis, and performance/security testing.

---

1. Testing Pyramid

        /\
       /  \        E2E Tests (few)
      /────\       UI automation, end-to-end flows
     /      \
    /────────\     Integration Tests (moderate)
   /          \    API tests, component interactions
  /────────────\   Unit Tests (many)
 /              \  Function/class/method level
/________________\

Level	Quantity	Speed	Maintenance Cost	Confidence
Unit Tests	Many (70%)	Very fast	Low	Low (single component)
Integration Tests	Moderate (20%)	Fairly fast	Medium	Medium (component interactions)
E2E Tests	Few (10%)	Slow	High	High (full workflow)

---

2. TDD (Test-Driven Development)

Red-Green-Refactor Cycle

1. Red:     Write a failing test
2. Green:   Write the minimum code to make the test pass
3. Refactor: Refactor the code while keeping tests green
↻ Repeat

Example

# Step 1: Red - Write the test
def test_calculate_discount():
    assert calculate_discount(100, 0.1) == 90.0
    assert calculate_discount(200, 0.25) == 150.0
    assert calculate_discount(100, 0) == 100.0

# Step 2: Green - Minimal implementation
def calculate_discount(price, discount_rate):
    return price * (1 - discount_rate)

# Step 3: Refactor - Add boundary checks
def calculate_discount(price, discount_rate):
    if price < 0:
        raise ValueError("Price must be non-negative")
    if not 0 <= discount_rate <= 1:
        raise ValueError("Discount rate must be between 0 and 1")
    return round(price * (1 - discount_rate), 2)

---

3. Unit Testing

3.1 pytest (Python)

import pytest
from myapp.user_service import UserService, UserNotFoundError

class TestUserService:
    @pytest.fixture
    def service(self):
        """Create a fresh service instance before each test method"""
        return UserService(db=MockDatabase())

    def test_get_user_by_id(self, service):
        user = service.get_user(1)
        assert user.name == "Alice"
        assert user.email == "alice@example.com"

    def test_get_nonexistent_user_raises(self, service):
        with pytest.raises(UserNotFoundError):
            service.get_user(999)

    @pytest.mark.parametrize("email,valid", [
        ("user@example.com", True),
        ("user@", False),
        ("", False),
        ("user@domain.co.uk", True),
    ])
    def test_email_validation(self, service, email, valid):
        assert service.validate_email(email) == valid

    def test_create_user(self, service):
        user = service.create_user("Bob", "bob@example.com")
        assert user.id is not None
        assert user.name == "Bob"

3.2 JUnit (Java)

@ExtendWith(MockitoExtension.class)
class UserServiceTest {

    @Mock
    private UserRepository userRepository;

    @InjectMocks
    private UserService userService;

    @Test
    void shouldReturnUserById() {
        when(userRepository.findById(1L))
            .thenReturn(Optional.of(new User(1L, "Alice")));

        User user = userService.getUser(1L);

        assertEquals("Alice", user.getName());
        verify(userRepository).findById(1L);
    }

    @Test
    void shouldThrowWhenUserNotFound() {
        when(userRepository.findById(999L))
            .thenReturn(Optional.empty());

        assertThrows(UserNotFoundException.class, 
            () -> userService.getUser(999L));
    }
}

---

4. Mocking

Mock objects replace real dependencies to isolate the unit under test.

from unittest.mock import Mock, patch, MagicMock

class TestOrderService:
    def test_place_order_sends_email(self):
        # Create mock objects
        mock_email_service = Mock()
        mock_payment_service = Mock()
        mock_payment_service.charge.return_value = True

        service = OrderService(
            email=mock_email_service,
            payment=mock_payment_service
        )

        service.place_order(user_id=1, amount=99.99)

        # Verify interactions
        mock_payment_service.charge.assert_called_once_with(1, 99.99)
        mock_email_service.send_confirmation.assert_called_once()

    @patch('myapp.services.requests.get')
    def test_fetch_external_data(self, mock_get):
        """Use patch to replace external HTTP calls"""
        mock_get.return_value.json.return_value = {"data": "test"}
        mock_get.return_value.status_code = 200

        result = fetch_data("https://api.example.com")
        assert result == {"data": "test"}

When to use mocks:

Scenario	Mock?
Database calls	Yes (in unit tests)
HTTP requests	Yes
File system	Depends
Pure functions	No
Internal methods	Usually not

---

5. Code Coverage

# pytest + coverage
pytest --cov=src --cov-report=html tests/

# Output
# Name                 Stmts   Miss  Cover
# ----------------------------------------
# src/models.py           45      3    93%
# src/services.py         78     12    85%
# src/utils.py            23      0   100%
# ----------------------------------------
# TOTAL                  146     15    90%

Coverage types:

Type	Meaning
Line coverage	Which lines were executed
Branch coverage	Whether each if/else branch was tested
Condition coverage	Each sub-condition in compound conditions
Path coverage	All possible execution paths

Coverage does not equal quality

100% coverage does not mean there are no bugs. Test quality depends on the effectiveness of assertions, not coverage numbers.

---

6. Code Review

6.1 Review Checklist

Dimension	Focus Areas
Correctness	Is the logic correct? Boundary conditions? Error handling?
Security	Input validation? SQL injection? XSS? Sensitive data?
Performance	N+1 queries? Unnecessary computation? Memory leaks?
Readability	Clear naming? Appropriate comments? Reasonable complexity?
Maintainability	Coupling? Single responsibility? Should it be split?
Testing	Are there tests? Do tests cover critical paths?

6.2 Best Practices

• Keep PR size within 200-400 lines
• Spend no more than 60 minutes per review session
• Provide constructive feedback; distinguish must-fix vs suggestions
• Automate what you can via CI (formatting, linting, tests)
• Focus on design and logic, not style (leave style to tools)

---

7. Static Analysis

7.1 Tools

Tool	Language	Function
pylint / ruff	Python	Code quality, style checking
mypy / pyright	Python	Static type checking
ESLint	JavaScript/TS	Code quality
SonarQube	Multi-language	Comprehensive quality platform
Bandit	Python	Security vulnerability checking
SpotBugs	Java	Bug detection

7.2 Example: ruff (Python)

# pyproject.toml
[tool.ruff]
line-length = 100
target-version = "py311"

[tool.ruff.lint]
select = [
    "E",    # pycodestyle errors
    "W",    # pycodestyle warnings
    "F",    # pyflakes
    "I",    # isort
    "N",    # pep8-naming
    "UP",   # pyupgrade
    "S",    # bandit (security)
    "B",    # bugbear
]

ruff check src/             # check
ruff check --fix src/       # auto-fix
ruff format src/            # format

---

8. Performance Testing

8.1 Types

Type	Purpose	Method
Load testing	Verify performance under expected load	Simulate normal user count
Stress testing	Find system limits	Increase load until failure
Soak testing	Check for long-running issues	Sustained moderate load over time
Spike testing	Verify burst traffic handling	Sudden large load increase

8.2 Tools

# Locust example
from locust import HttpUser, task, between

class WebsiteUser(HttpUser):
    wait_time = between(1, 3)

    @task(3)
    def view_items(self):
        self.client.get("/api/items")

    @task(1)
    def create_order(self):
        self.client.post("/api/orders", json={
            "item_id": 1,
            "quantity": 2
        })

# Run Locust
locust -f load_test.py --host=http://localhost:8000
# Or headless mode
locust -f load_test.py --headless -u 1000 -r 50 --run-time 5m

Key metrics:

• Response time: P50, P95, P99
• Throughput: RPS (Requests Per Second)
• Error rate: percentage of failed requests
• Resource usage: CPU, memory, network, disk I/O

---

9. Security Testing

9.1 OWASP Top 10

Rank	Threat	Protection
1	Broken access control	Least privilege, deny by default
2	Cryptographic failures	TLS, secure hashing, key management
3	Injection	Parameterized queries, input validation
4	Insecure design	Threat modeling, secure design principles
5	Security misconfiguration	Minimal installation, hardened configuration
6	Vulnerable components	Dependency scanning (Dependabot)
7	Authentication failures	MFA, secure password policies
8	Data integrity failures	Signature verification, CI/CD security
9	Logging and monitoring failures	Centralized logging, alerting
10	SSRF	URL allowlisting, network isolation

9.2 Security Testing Tools

Type	Tool	Description
SAST (Static)	Bandit, Semgrep	Analyze source code
DAST (Dynamic)	OWASP ZAP, Burp Suite	Runtime scanning
SCA (Dependencies)	Dependabot, Snyk	Dependency vulnerabilities
Container	Trivy, Clair	Image security scanning
Secrets	TruffleHog, GitLeaks	Detect leaked secrets

---

Relations to Other Topics

• See Version Control and CI/CD for how testing becomes part of continuous integration and delivery
• See System Design for how quality attributes must be designed at the architecture stage
• See Software Engineering Overview for the role of validation in the software lifecycle
• See Full-Stack Development for how frontend, backend, and end-to-end tests work together

References

• "Unit Testing Principles, Practices, and Patterns" - Vladimir Khorikov
• "Clean Code" - Robert C. Martin
• OWASP Testing Guide: https://owasp.org/www-project-testing-guide/
• pytest Official Documentation: https://docs.pytest.org

---

评论