Modular Reporting & Analytics

Testform’s reporting engine provides an enterprise analytics and reporting platform designed with architectural decoupling principles. It processes the execution ledger (testform.state) and renders multi-dimensional metrics across several output formats.

---

Engine architecture

The reporting system divides responsibilities into three distinct layers:

┌─────────────────────────┐
│     testform.state      │
└────────────┬────────────┘
             │
             ▼
┌─────────────────────────┐
│   StateQuery Engine     │ ◀── -filter "key=val", free text, -groupBy
└────────────┬────────────┘
             │
             ▼ (ReportContext)
┌─────────────────────────┐
│    IReport.collect()    │ (Core: 9 reports / Custom: .testform/reports/*.ts)
└────────────┬────────────┘
             │
             ▼ (ReportData)
┌─────────────────────────┐
│  IReportFormat.render   │ (md, json, csv, xlsx, docx / *.format.ts)
└────────────┬────────────┘
             │
             ▼
┌─────────────────────────┐
│     stdout or File      │ (-out=...)
└─────────────────────────┘

1. Query & Normalization (StateQuery): Filters state items respecting tags, scenario attributes, execution outcomes, and custom fields declared in main.yml. Emits a normalized QueryResultItem[] collection.
2. Extraction & Aggregation (IReport): Accepts the execution context (ReportContext) and calculates high-level metrics, headers, and tabular rows (ReportData).
3. Rendering & Interpolation (IReportFormat): Projets the aggregated data into the desired output format, interpolating ${...} expressions and applying patches to binary or textual templates.

---

Dynamic templates in Word & Excel

Testform allows customizing the visual layout of executive reports using preconfigured templates. To maintain complete consistency with Testform’s variable engine and avoid syntax collisions, all templates use standard ${...} placeholders.

Interpolation placeholders

Placeholder	Description	Injected Value
${var.<name>}	Configuration or CLI variable.	Evaluated against vars.yml or -var.
${report.title}	Title generated by the active report.	E.g., Test Run Summary
${date}	Generation timestamp.	Formatted date and time string.
${<metric>}	Key-value report metric (total, passed, etc.).	Numerical value or percentage string.
${table} / ${report.table}	Table insertion anchor.	Substituted with the complete generated dataset table.

Creating a Word template (.docx)

1. Open Microsoft Word and design your corporate header, covers, or branding.
2. Add paragraphs containing template placeholders:
   • Project: ${var.project_name}
   • Report: ${report.title}
   • Generated: ${date}
3. In the location where the data table should be placed, add the text ${table} in an independent paragraph.
4. Save the file as report.docx inside .testform/reports/ (used automatically) or supply it via -template=path.docx.

Creating an Excel template (.xlsx)

1. Open Excel and format header rows and company branding.
2. In metadata cells, add expressions like Environment: ${var.environment} or Passed: ${passed}.
3. In the exact starting cell where the data table should render (e.g. B10), enter ${table}.
4. The engine locates coordinate (row, col), renders corporate headers, and inserts the data rows with auto-fitted column widths.

---

Extensibility and autoloading

In line with Terraform’s design principles, extending Testform requires zero configuration overhead in main.yml. Extensions work by convention inside .testform/reports/:

my-project/
├── .testform/
│   └── reports/
│       ├── report.docx              # Default project Word template
│       ├── report.xlsx              # Default project Excel template
│       ├── team-velocity.ts         # Custom report: "testform report team-velocity"
│       └── slack.format.ts          # Custom format: "testform report ... -format=slack"
├── main.yml
└── tests/

Example 1: Creating a Custom Report

Create .testform/reports/team-velocity.ts:

import type { IReport, ReportContext, ReportData } from '@fromeroc9/testform';

export const TeamVelocityReport: IReport = {
  name: 'team-velocity',
  description: 'Team testing productivity and velocity metrics',

  async collect(ctx: ReportContext): Promise<ReportData> {
    const total = ctx.items.length;
    const passed = ctx.items.filter(i => i.status === 'passed').length;
    const team = ctx.variables['team'] || 'QA Team';

    return {
      title: `Quality & Velocity - ${team}`,
      metrics: {
        'Team': team,
        'Total Scenarios': total,
        'Passed': passed,
        'Effectiveness': total > 0 ? `${Math.round((passed / total) * 100)}%` : '0%'
      },
      headers: ['ID', 'Scenario', 'Assignee', 'Status'],
      rows: ctx.items.map(item => [
        item.id,
        item.title,
        item.assignee || 'Unassigned',
        item.status
      ])
    };
  }
};

export default TeamVelocityReport;

Run the custom report immediately:

testform report team-velocity -format=md
testform report team-velocity -format=docx -out=velocity.docx -var="team=Core Banking"

---

Example 2: Creating a Custom Output Format

Create .testform/reports/slack.format.ts:

import type { IReportFormat, ReportData, ReportContext } from '@fromeroc9/testform';

export const SlackFormat: IReportFormat = {
  name: 'slack',
  extension: '.json',

  async render(data: ReportData, ctx: ReportContext): Promise<string> {
    const blocks = [
      {
        type: 'header',
        text: { type: 'plain_text', text: `${data.title}` }
      },
      {
        type: 'section',
        fields: Object.entries(data.metrics || {}).map(([k, v]) => ({
          type: 'mrkdwn',
          text: `*${k}:*\n${v}`
        }))
      }
    ];

    return JSON.stringify({ blocks }, null, 2);
  }
};

export default SlackFormat;

Generate any report with the new format:

testform report testrun-summary -format=slack -out=dist/slack-payload.json

---

Supported native formats

Format	Flag	Extension	Type	Recommended Use Case
Markdown	-format=md	.md	Plain text	CI/CD consoles, GitHub Actions Step Summary, terminal output.
JSON	-format=json	.json	Structured text	Dashboard ingestion, REST APIs, downstream automation scripts.
CSV	-format=csv	.csv	Tabular text	Simple spreadsheet import, data pipelines, BI utilities.
Word	-format=docx	.docx	Formatted binary	Executive compliance audits, formal client deliverables.
Excel	-format=xlsx	.xlsx	Structured binary	Quantitative analysis, complex coverage matrices.

On-Demand Binary Drivers

To maintain Testform’s minimal package size, binary generators are optional peer dependencies:

• Word: npm install -g docx (or npm install --save-dev docx)
• Excel: npm install -g exceljs (or npm install --save-dev exceljs)