POLISH holds that a system needs continued maintenance after launch. As long as people still use a system, the way it is used, its runtime environment, and its security requirements keep changing. A system that is not adjusted to these changes gradually becomes hard to use and hard to modify. POLISH divides post-launch work into fixed monthly routine work and exceptional work done to meet various requirements, and records both so that the project team and the supervising agency can verify the work at any time. This article is the hub of the long-term maintenance and government collaboration series, and later sections link to the dedicated articles in the series.
Software evolution and how maintenance work is classified
The paper by Lehman, 1980 proposes the law of continuing change: a program that is in use and has to reflect reality becomes progressively less useful if it is not continually adjusted, until it is replaced by a rebuild. The same paper points out that when changes are made without maintenance effort, the program structure deteriorates and its complexity grows.
The questionnaire in Lientz et al., 1978 was sent to 120 organizations, and 69 responses came back. The responses show that maintenance and enhancement take up most of the resources of systems and programming development groups, and that management considers maintenance and enhancement at least slightly more important than new development. Maintenance problems lean toward the managerial side, and user requests for enhancements and extensions are the most important management problem.
POLISH handles maintenance as a management task: each item is listed on a checklist, carried out on a fixed cycle, and recorded.
Traditionally, all work after delivery is called maintenance and is often divided into four types: perfective, adaptive, corrective, and preventive. Rajlich and Bennett, 2000 argue that post-delivery work is in fact a series of distinct stages, each with different activities, tools, and business consequences, and that both industry and engineering can benefit from understanding these stages and the transitions from one stage to the next.
Swanson, 1976 compares software maintenance to an iceberg, with much below the waterline that is hard to measure. Such work needs to be recorded so that the project team can check it.
In the same checklist, POLISH separates monthly routine work from exceptions. Routine work follows a fixed rhythm, and exception events are entered into that month's maintenance items.
| Type | Content | Record kept |
|---|---|---|
| Routine | Monthly collection of the data needed for the maintenance report; account reviews, backup restoration drills, and personal computer security checkups on the annual schedule | Maintenance report in a format the project team accepts; records and evidence from drills and reviews |
| Exception | Changes made to comply with security notices, vulnerability scans, or security audits | Entered into that month's maintenance items, with item-by-item handling notes |
- Maintenance work for the month
- Determine where the work comes from
- Fixed monthly data and reportRoutineCarried out on a fixed cycle, with records kept
- Compliance with security notices, vulnerability scans, or security auditsExceptionEntered into that month's maintenance items, with handling notes
- A problem occurs in the systemIncident handling processReporting, repair, notification, and post-incident report
Compiled from the monthly maintenance report practice. When a problem occurs in the system, the incident handling process applies separately.
Besides the items in the table, problems can also occur in the system. When a problem occurs in the system, POLISH will assign staff to handle it immediately. The handling process is described in how system incidents are reported, classified, and repaired, and how severity levels and targets are agreed is covered in how to write SLA severity levels and targets.
Routine security work runs on an annual schedule
Routine security work needs a predictable schedule. For each quarter, POLISH opens a discussion thread for routine security work, covering restoration drills, account reviews, and personal computer checkups. Before starting, POLISH confirms the schedule and scope with the project team, then carries out the work on the annual schedule and keeps records.
How meetings, reporting checklists, and launch checks are arranged for an annual project is covered in the communication cadence of annual projects across agencies. The day-to-day practice of handling reports and periodic checks is covered in routine maintenance after launch.
Handling runtime updates and scan results
POLISH works with the Ministry of Education project, carrying the work from the design tender to the engineering tender. After the Outdoor Education Resource Platform went live, POLISH has supported the scans, the patching of medium- and high-risk findings, and the false-positive appeals that the supervising agency requires. The practices below apply to updates and scan handling in general.
Official security fixes for the runtime environment and packages need to be adopted promptly. After a security fix release of the Node.js runtime, POLISH proactively reminds the project team and arranges the upgrade. Where the host side must cooperate, the project team passes the request to the host's operations contact, and once everything is in place the upgrade is completed and verified item by item.
The monthly routine host scan is handled by the deadline the supervising agency sets. After receiving the scan report, the project team notifies the lead engineer and forwards the report. Engineers make the fixes by the specified deadline and reply when handling is done. The project team then enters the handling notes into the remediation progress table and reports to the supervising agency.
Compiled from the deadlines the supervising agency specifies.
When only server settings are adjusted and the code is unchanged, the system change form does not have to be filled in at first, but a brief description of the adjustment is still required so that the project team can complete the form. How to prioritize patches after a scan report arrives and how to appeal false positives are covered in what to do after a vulnerability scan report arrives.
The supervising agency may also, based on the results of an external security comparison, issue a list of systems that need software patched, updated, or removed, with a deadline for reply. Engineers handle the list item by item and state what was done. The project team compiles the results and replies to the supervising agency, which uses the reply to sign off the items.
Regulatory and security requirements change every year
Public sector security requirements are adjusted every year, so a system needs to be reviewed even when its features have not changed. The annual penetration test and vulnerability scan are arranged by the supervising agency, and POLISH supports patching and retesting.
The annual penetration test proceeds in stages: a scope-of-work briefing comes first, followed by testing in rounds, then a briefing on the initial test report, and after fixes are made, a briefing on the retest report. At each stage the project team reposts the schedule and materials in the fixed communication channel, and anyone who could not attend can watch the recording and slides afterward. The difference between the two kinds of testing and the legal frequencies are covered in how a vulnerability scan differs from a penetration test.
Requirements also land on concrete settings. The lead engineer observed that government security checklists had mostly raised the minimum password length from 8 to 12 characters, informed all developers, and changed the password rule to at least 12 characters with upper- and lowercase letters and digits. The front-end prompt text was then synchronized with the back-end rule, with unified wording provided by the lead engineer, and an explanatory popover was added afterward.
To meet the architecture requirement in that year's security regulations, POLISH adjusted the host architecture and notified the project team. Because the improvement measures required auditable evidence, POLISH engineers promptly captured the evidence screenshots.
The security enhancement measures and self-check list compiled by the supervising agency cover categories such as input and output handling, security header configuration, file upload validation, and password storage. After the project team reposted the list, the engineering team went through the self-check list item by item and verified each item directly. A list that can be checked item by item gives routine requirements a basis for review.
Public sector use of artificial intelligence will also be inventoried and audited in the future. Several months before the relevant check requirement appeared, POLISH had already placed the public sector AI application reference manual in the team's shared materials for reading. When the checklist arrived, POLISH inventoried the parts of the project that actually use AI. The next day, POLISH submitted a filled-in version and reminded the project team to review it, and the project team completed the submission that day.
How technical debt builds up and is repaid
Technical debt is a metaphor used by Cunningham, 1992: shipping code that has not yet been cleaned up is like taking on debt, which speeds things up in the short term and is harmless if the code is rewritten promptly to pay it off. When the debt is not repaid, every bit of time spent on code that is not good enough counts as interest, and an entire engineering organization can be dragged to a halt by an unconsolidated implementation.
Sculley et al., 2015 point out, for machine learning systems, that rapid progress appears to cost nothing but in practice accumulates substantial long-term maintenance cost, and that data dependencies are one source of risk. For a system with AI features, the scope of maintenance therefore includes a data layer. The refusal thresholds and abuse defenses after an AI question-answering system goes live are covered in how to keep an AI Q&A system trustworthy after launch.
Several months before a system went live, POLISH had the lead engineer list the basic security practices and asked the developers to follow these practices:
- Put all keys and similar settings in environment variables
- Add a fixed identifying header to requests
- Attach an anti-forgery token to every write request
- Wrap access to login credentials in a separate module
- Define the rule for automatic logout after idle time first
Of these, wrapping access to login credentials in a separate module makes the system easier to adjust when a different authentication method is adopted later.
How system knowledge carries over when design and engineering fall under separate tenders is covered in a dedicated article: how system knowledge continues from the design tender to the engineering tender.
How to write a maintenance report
A maintenance report lets the project team and the supervising agency check what was done in the month. Every month, POLISH collects the data that the maintenance report requires and produces the report in a format the project team accepts. The report is organized in this order:
- Fixed monthly data: the data required for the maintenance report, provided in a format the project team accepts.
- Exceptions for the month: changes made to comply with security notices, vulnerability scans, or security audits are entered into that month's maintenance items.
- Item-by-item handling notes: what was done on each item is stated; when only server settings were adjusted and the code is unchanged, a brief description of the adjustment is provided as well.
- Evidence: drill records and screenshots, kept before the drill environment is cleaned up.
Requirements, decisions, and launch notifications stay in the shared communication channel. The practice is covered in how to keep requirements and decisions on record in a shared communication channel.
For improvements beyond the agreed scope, POLISH writes a proactive proposal and validates it with a POC first, and only then decides whether to bring it into the agreement. The practice is covered in POC work beyond the agreed scope.
Situations where this approach does not fit
The routine schedule and item-by-item records in this article apply to systems in long-term use that are bound by the security requirements of a supervising agency. A short-term campaign page is taken down once the campaign ends, so the maintenance focus is handling during the campaign and data retention afterward, and no annual schedule is needed. For an internal tool with no external audit requirement, the organization that uses it can set the cycle of routine work and does not need to adopt the frequency of drills and reviews.
What to prepare before starting
Before migrating to a managed environment, POLISH sorted the questions to confirm with the host's operations contact into three categories on a checklist, so that no gap is discovered halfway through:
- Servers: host configuration for the test and production environments, whether remote connection is possible, and whether a request form is needed
- Security: who arranges vulnerability scans and penetration tests, in which environment the testing runs, and who bears the cost
- Deployment: which checks must be completed before an existing website is migrated, and who deploys the production environment
Contract and acceptance are also aligned at this stage. A long-term outsourcing arrangement has to manage the contract terms and the relationship between the parties together, as covered in contract terms and relational governance in long-term outsourcing. How acceptance metrics map to the client's goals is covered in how to write acceptance from requirements to outcome metrics.
POLISH treats the client's goals as its own goals, and maintenance after launch likewise starts by aligning responsibilities, environments, and the way records are kept. The service is described in long-term maintenance and platform development. To assess the current state of a system and the scope of maintenance, gather the supervising agency's security requirements and any existing scan reports, then contact POLISH.
