From Design Tender to Engineering Tender: How Public-Sector Projects Carry System Knowledge Forward

Working on the Ministry of Education project, POLISH carried the Outdoor Education Resource Platform from the design tender into the engineering tender, with the same team responsible for the design phase and the engineering phase. Code and documents stay behind with the tender, but understanding of the system is held by the people who handled it. That understanding is the most likely thing to break when a tender changes phase.

System understanding is held by the people who handled it

This view comes from Naur, 1985. The paper argues that the essence of programming is the programmer forming a theory of the problem, which is to say a deep understanding of it. Documents cannot record that understanding in full. When the team that holds the theory is dissolved, the program text can still run, but it can no longer respond sensibly to requests for modification. A new person cannot acquire the theory by reading code and documents alone and needs to work closely with people who already hold it.

The impact of staff turnover on a project can be measured in numbers. Avelino et al., 2016 propose a method for estimating TF automatically. TF is the number of developers whose departure would leave the project unable to continue, and it measures how concentrated knowledge is. The researchers ran the estimate on 133 popular GitHub projects, and 65% of the projects had a TF of 2 or less. The sample consists of popular open-source projects.

What the design phase leaves as a basis for engineering-phase judgment

Regulation places this matter among the considerations for agency outsourcing. Article 7 of the Enforcement Rules of the Cyber Security Management Act provides that, when an agency outsources the building or operation of an information and communication system, the selection and supervision of the contractor should consider staffing, subcontracting, security testing of custom-built systems, incident reporting, and the handling of data when the contract ends. The official FAQ of the Administration for Cyber Security, Ministry of Digital Affairs further recommends stating the requirements for verification and for accepting audits in the tender documents in advance, to avoid disputes during performance.

POLISH maintains the platform planning and process design files produced in the design phase as documents that the engineering phase will still need to consult. The project team needed the earlier platform planning and process design files to answer a higher-level authority's questions about new features. POLISH prepared the files and checked whether the files needed updating, completed the update the same day, and the project team then pasted the file links back into the discussion thread for the record. Questions in the new phase can therefore be checked against existing design and process decisions.

How data and users from the existing system are carried over

Before the old and new systems are handed over, POLISH first confirms how the data in the existing system can be brought out. The practice recorded in the meeting minutes is that the current back end can export data, so there is no need for now to approach the maintainer of the original system separately. If contact is really needed later, the project team acts as the intermediary contact.

Transferring existing data and users requires cooperation from the maintainer of the original system. The project team judged that the meeting with that party was better scheduled after the tender, and POLISH accepted this and moved that part of the schedule back. On the data side, a fallback was prepared at the same time: ask the other party to export the existing database, and in the worst case export tables from the back end.

Once the data is brought out, there is still the work of moving it. When more than three hundred sites were moved from the existing system into the new back end, POLISH divided the data by county and city into four groups, north, central, south, and east plus the outlying islands, with tens to over a hundred sites per group, and each group was claimed by the people doing the transfer. The transfer procedure was written as three steps: look up the site in the existing system, paste it into the new back end as a new entry, then fill in the detailed information and images. Gaps in the data were looked up and filled in by the same people, and comments on the back-end workflow that came up during the transfer were recorded in a quality list. When the input format of the back end was adjusted, the change was announced in the shared communication channel with example screenshots attached.

Published entry points and document links also have to be carried over. When the website's announcement categories were merged, POLISH first asked that the homepage links that had pointed to the old categories be redirected to the new category. The engineer kept the existing category IDs, changed the URLs first, and then confirmed that operation was normal; after the change, links to the old category redirected to the new one and worked normally. Before the old and new systems were switched, POLISH also reminded the team that the URLs of existing documents should be set up in advance and replaced with URLs of the new system once it had been updated.

On the day of the switch, the engineer explained the differences between the old and new systems to the project team. In both, content can be edited from the back end, and web page content is stored in a database. The new system was developed entirely from scratch and reuses no code from the existing system, so its mechanism of operation differs.

Decisions and forms are kept in one place

POLISH fixes the place where decisions are stored. Discussion records on hosting and information security are managed by POLISH in one shared spreadsheet and one file folder, and the project team updates the spreadsheet directly. Every decision therefore has a source that can be traced back.

Event and course data likewise has a single source. The project team maintains the data in a shared master sheet and reports to POLISH at the regular meeting after the monthly update. When the regular meeting is suspended, the report is made in the discussion thread instead, and POLISH imports the data from the master sheet. A single data source prevents the two sides from each editing separately.

The options offered in the interface are also settled in writing. Category options are defined on the interface side, and the engineering side learns of changes only if told in advance, so POLISH asks proactively when the options need to be finalized, requests that the other side compile a list in writing, and asks that any later change be announced first. Once an option is modified, all related data is checked again.

Items with no ready-made data source need clearly stated rules first. Teachers choose a school when registering, and no ready-made school list API is available, so the list can only be compiled manually into a format the platform can use. POLISH first wrote out the rules for special cases: schools covering junior high through senior high are placed on both lists, five-year junior colleges get a separate tab, foreign schools are placed according to the grade levels actually offered, and any school that cannot be categorized is ignored for now. The project team copied entries from the public lists into a spreadsheet, divided into university, senior high, junior high, and elementary, and returned the finished spreadsheet, and POLISH took over the processing.

Annual routines continue through retained originals

Forms related to information security, scan and test reports arranged by the supervising agency, and audit evidence recur every year. POLISH keeps these materials in the master project sheet, the information security communication page, and a dedicated evidence folder. The project team logs newly received reports in the spreadsheet the same day, and when POLISH needs to look up past applications and records, it asks the project team for the version retained last year. With a retained original for the annual information security routine, nothing has to be searched for from scratch each year. For the evidence the annual information security work leaves behind, see delivery and operations evidence in public-sector projects.

When the annual guidelines change, the existing back end is likewise reused. After the supervising agency approves the annual guidelines, the project team pastes the guidelines to POLISH for planning reference, and POLISH then asks the project team to share the course and activity lists of each unit. The project team provided a cloud folder and explained that 20 courses had already been listed in the back end; course data for the new year reuses the existing back end, with years separated by date.

Where new team members get an understanding of the system

When taking over an existing system, POLISH confirms the current state first and discusses changes afterward. Information security maintenance reports need traceable data, so POLISH lists the preparation work from 2 perspectives: building from scratch and taking over an existing system. Building from scratch first calls for automatic backup settings, completeness of the logging mechanism, and whether code can be written to obtain report data quickly. Taking over an existing system first calls for confirming the software installed on the server and how it is deployed, the scope and storage path of backups, the logging settings and storage path, and whether a data-retrieval script already exists. For the full takeover procedure, see the full code audit process for taking over an existing system.

New staff on the agency side taking over the platform involves knowledge transfer. Ko et al., 2005 use paired consultant and client data from 96 ERP implementation projects to show that clients expect consultants to transfer implementation knowledge to employees so that the system can later be maintained without the consultant. Knowledge transfer is affected by three kinds of factors: knowledge, motivation, and communication. The research setting is enterprise system implementation.

Naur holds that rebuilding this understanding from documents alone is at best a high-cost approach. POLISH has the same team carry the work from the design tender to the engineering tender, so new members can ask the people who handled the work directly and then check against the records kept in the earlier sections.

How tidy the code is also affects whether new members can get started. The experience report by Cunningham, 1992 uses a debt metaphor: shipping code that is not yet tidied is like going into debt, which speeds things up in the short term, and if it is not repaid, every minute spent on not-quite-right code counts as interest. The report also notes that excessive untidiness makes code hard to grasp and leads to excessive specialization among programmers. For the relationship between technical debt and operations after launch, see why maintenance continues after launch.

New members can get an understanding of the system from the following four places:

• Platform planning and process design files from the design phase
• Decision records in the shared spreadsheet and file folder
• Applications, reports, and evidence retained from last year
• The engineer's explanation of the differences between the old and new systems on the day of the switch

Situations where this does not fit

This article addresses situations where the system will keep being modified and the tender changes phase. The need for knowledge continuity is lower in the following situations:

• Event websites that are taken down after the event, with no modification needs afterward
• Small systems with simple functions, where post-launch changes continue to be handled by the original maintainer
• Agencies with an in-house team that already provides ongoing operations and stays involved throughout a change of phase

In these situations, the handover focuses on data export and account transfer.

What to prepare before starting

When a project moves from the design tender to the engineering tender, or an existing system is to be handed to a new team for operations, POLISH's long-term operations and platform development begins by comparing the current state of the following items with the project team.

• How data can be exported from the existing system, and the fallback approach
• Actions that need to be applied for separately, and the contact who actually helps
• The shared spreadsheet and folder where decisions and discussion records are kept
• Retained versions of the previous round's applications, scan and test reports, and audit evidence
• Whether the platform planning and process design files need updating
• For items with no ready-made data source, who sets the compilation rules, who compiles, and who takes over

POLISH treats the client's goals as its own, and the same team stays responsible when the phase changes. When a project is about to change phase or be handed to a new team, the current state of the items above can be compiled first, after which the project's stage can be described at discussing handover and long-term operations with POLISH.