Auditor-General’s Report
Results of special reviews and other investigations, May 2005

5. IMPLEMENTATION OF THE TELECOMMUNICATIONS PURCHASING AND MANAGEMENT STRATEGY (TPAMS)

5.1 Audit conclusion

In July 2002, as part of its ‘eGovernment vision - Putting People at the Centre’ initiative, the government launched the Telecommunications Purchasing and Management Strategy (TPAMS). The strategy establishes a framework for the whole-of-government procurement and management of telecommunications services into the future. It involves 4 projects that aim to improve the accessibility, quality and cost of telecommunication services across the Victorian public sector.

Our examination of the implementation of TPAMS concludes that, overall, this initiative was effectively planned and managed. Although minor project delays and project cost revisions were experienced, the project is still on track to meet the June 2006 deadline and is within budget. However, enhancements to project cost and milestone reporting need to be considered.

The program’s project governance and management by the Department of Infrastructure (DOI) and its Chief Technology Office (CTO) is robust. However, the role of the Office of the Chief Information Officer (within the Department of Premier and Cabinet) needs to be reviewed and clarified in relation to monitoring whole-of-government information technology projects.

The telecommunications contract procurement process was assessed as fair and followed due process and complied with Victorian Government Purchasing Board guidelines.

It is still too early for us to conclude that all TPAMS program expected benefits and cost savings will be achieved. However, early indications are that lower telecommunication prices were negotiated and regional areas may have enhanced telecommunications access. The project team also indicated that the original expected savings, of $73 million over 5 years, are likely to be exceeded.

The government’s intention is that the savings achieved from the project will be re-invested in additional bandwidth or telecommunication services, and therefore provide a greatly improved service. However, this will be achieved at a cost of $55.2 million.

Although the TPAMS program risks are adequately managed, as the TPAMS program progresses risks associated with government department and agency take up of telecommunication services and the chief technology office’s ability to manage multi-supplier and user telecommunication services, will need to be closely managed.

RESPONSE provided by Secretary, Department of Infrastructure

The Department of Infrastructure administers the Telecommunications Purchasing and Management Strategy (TPAMS) which establishes a framework for the whole-of-government procurement and management of telecommunications services.

The Auditor-General’s report acknowledges the importance of this strategy which, through a number of projects, aims to improve the accessibility, quality and cost of telecommunication services across the Victorian public sector.

The report presents 4 key recommendations. The department accepts and is supportive of the report’s recommendations.

5.2 Background

The Victorian government is the largest single purchaser of voice, data and mobile telephony telecommunications in Victoria, spending about $178 million a year.

In March 2002, the government launched its ‘eGovernment vision - Putting People at the Centre’ policy. In July 2002, as part of this policy, it launched the Telecommunications Purchasing and Management Strategy (TPAMS). The strategy establishes a framework for the whole-of-government procurement and management of telecommunications services into the future.

The objectives of TPAMS are to improve:

• competition, to lower the cost of telecommunications to the government and businesses and to provide access to the latest technology

• connectivity between agencies

• convergence, to provide government data, voice and video services through a single network.

TPAMS aims to replace the current Victorian government telecommunications framework, which consists of:

• the StateNet telephone facilities management contract, which services 23 000 users in the Melbourne central activities district and at several regional locations

• the VicOne data services contract, which extends to some 4 500 sites across the state

• the Telecommunications Purchasing Strategy, which involves the use of 4 telecommunication service panels.

Until October 2003, TPAMS was managed by the executive director, Multimedia Victoria, in the Department of Infrastructure (DOI). From that date, the Chief Technology Office of DOI assumed responsibility for managing the implementation of TPAMS and the executive director, Multimedia Victoria, assumed the role of project sponsor.

Strategy implementation is expected to cost $55.2 million over 4 years, and result in savings to departments and agencies in excess of $73 million over 5 years. Savings are expected to result from reduced prices (call costs) as a result of increased competition between suppliers.

At the date of preparing this report, all procurement projects under the strategy had started. The projects are:

• Victorian Office Telephony Services (VOTS)

• Telecommunications Carriage Services (TCS) (for data, voice and mobile services)

• Local Area Network (LAN) Transport Upgrades

• Network Operations Support (NOS)

Under the strategy, DOI has a head agreement with each supplier that establishes the broad framework and cost structures under which services are to be provided to government agencies, and a service agreement that outlines the technical specifications which are to be met by the supplier. The departments and agency users of the telecommunication service then have customer contracts that specify the services to be provided and the associated cost for these services. Figure 5A illustrates the telecommunications procurement framework.

FIGURE 5A: TELECOMMUNICATIONS PROCUREMENT FRAMEWORK

Our audit examined how well the implementation of TPAMS was being managed. In doing so, we assessed whether:

• a robust governance structure was established over the project

• the contract procurement process was well conducted, fair and competitive

• the project was on time and within budget

• the expected savings and other benefits from TPAMS were realised

• the project risks were adequately managed.

5.3 Does TPAMS have a robust governance structure?

In assessing whether TPAMS has a robust governance structure, we examined if:

• roles and responsibilities were clearly allocated

• there was regular monitoring and reporting to a project governance committee

• probity and audit arrangements were adequate

• project planning was sound.

Figure 5B shows the governance structure adopted for the management and monitoring of the whole strategy and its individual projects. The structure comprises the project control board (represented by the CTO of Multimedia Victoria), governance committee, stakeholders reference group and the reporting channels to Multimedia Victoria and the Secretary of DOI. A number of independent assurance parties (that is, internal auditors and probity auditors) have input into the governance arrangements.

FIGURE 5B: TPAMS PROGRAM GOVERNANCE & MANAGEMENT STRUCTURE

Source: Multimedia Victoria and Chief Technology Office.

The project sponsor (Multimedia Victoria), the governance committee, the project management office, and the expected users of the systems all have opportunity for input.

The governance committee¹, established in June 2003, meets regularly to assess the project management team’s risk management and project progress.

The project control board (which consists of representatives from Multimedia Victoria and the CTO) makes regular and structured reports about the progress of the projects to the project sponsor and to the governance committee, using a balanced scored card² approach.

A probity auditor was employed to ensure that tender evaluation processes were fair. The projects were also subject to audit by the department’s internal auditors.

The project management teams used the government’s preferred Prince2 project management methodology, and used specialist services where necessary. A comprehensive method (supported by software) of reporting and monitoring of project risks was also established.

While the established governance arrangements are quite comprehensive, we observed the following minor weaknesses that require attention:

• The Office of the Chief Information Officer (OCIO) was not involved, with the CTO, in the day-to-day management of the projects. This needs to be addressed by the OCIO accepting a more formal role in monitoring and reviewing project progress. It is acknowledged that the CTO only became responsible for the oversight of TPAMS, from a whole-of-government perspective, from October 2004.

• The project governance committee (which has responsibility for monitoring the project’s progress) only monitors capital expenditure, leaving the monitoring of operating expenditure to the project sponsor, Multimedia Victoria.

Conclusion

The TPAMS governance structure is robust. There is input by a number of high-level committees, and independent assurance at various stages of projects. However, the governance structure could be improved by clarifying the monitoring role of the OCIO and by the governance committee monitoring capital and operating expenditure on each individual project.

Recommendations

1. That the TPAMS governance committee monitor whole-of-project-life costs, and capital and operating expenditure, for each of the TPAMS projects.

2. That the OCIO and other central government agencies interested in whole-of-government information and communication technology (ICT) projects review and clarify their monitoring role and reporting responsibilities for large ICT projects, such as TPAMS.

RESPONSE provided by Secretary, Department of Infrastructure

Recommendation 1 agreed. Whole-of-project-life costs should be monitored. A governance review will now be conducted as one of the next steps in a recently completed TPAMS major procurement project. The Auditor-General’s recommendation will be considered as part of that review. This whole-of-project-life approach will ensure that capital expenditure is monitored during the project delivery phase. Operating expenditure continues to be managed in line with VPS budget management processes.

Recommendation 2 agreed.

RESPONSE provided by Acting Chief Information Officer, OCIO

The Office of the Chief Information Officer (OCIO) made the following recommendations to the ICT (Information and Communications Technology) Strategy Sub-Committee of the ERC (Expenditure Review Committee) of Cabinet at its meeting of 17 February 2005. These recommendations were accepted. They relate to ICT projects that receive funding from the ERC (both in past years and in future):

• that ERC ICT projects be asked to report to the OCIO with frequency determined through a risk-based assessment by the OCIO

• that the OCIO develops a proposal, for future consideration, to create a Benefits Realisation initiative that will:

• develop overarching policy, guidelines and principles

• develop a Benefits Realisation Management methodology encompassing framework, register, implementation plan and training programs

• assign central responsibility for co-ordination, monitoring and reporting.

The OCIO has defined the reporting frequency for ICT projects that received ERC funding in past years. These projects are to be asked to forward their normal reports, which are expected to cover delivery against the target schedule, progress against budget and risk management.

The OCIO plans to have a Benefits Management proposal ready for the ICT Strategy Sub-Committee by the end of July 2005.

RESPONSE provided by Secretary, Department of Premier and Cabinet

The department acknowledges your recommendation 2 that “The Office of the Chief Information Officer review and clarify their monitoring role and reporting responsibilities for large ICT Projects, such as TPAMS”.

5.4 Was the contract procurement process fair and competitive, and did it follow due process?

In assessing whether the contract procurement process was fair and competitive, and whether it followed due process, we examined if it was conducted in line with Victorian government purchasing guidelines and was supported by an adequate probity audit process. The guidelines require:

• fairness and impartiality

• the use of a competitive process

• a consistent and transparent process

• confidentiality

• conflicts of interest to be identified and resolved

• compliance with legislative obligations and government policy.

We also examined how the contract processes were managed, and if contracts defined user requirements and deliverables that could be measured and monitored.

NEC Business Systems was awarded the contract to operate and update the government’s CBD internal office telephone system in April 2004. Telstra and Optus were awarded contracts for the first tranche of fixed voice, data and mobile phone services in September 2004. The probity auditor assessed the 2 procurement processes as fair, and considered that the team that evaluated tenders had followed the Victorian government purchasing guidelines. DOI’s internal auditor and an independent contractor also found that the processes were fair and competitive.

We also found that the Victorian Government Purchasing Board guidelines were complied with. Specifically, we found compliance with the guidelines for Victorian industry participation, advertising, submission requirements and approval by DOI’s Approved Purchasing Unit. There was an adequate contract management plan, adequate resourcing, risk management policies, probity policy and audit, and transition arrangements.

We reviewed the request for tender documents and found that all specified evaluation criteria and the scope of financial information required, encouraged innovative solutions, identified the controls and ensured fairness and efficient processing, and clear instructions for delivery and lodgment.

Conclusion

The TPAMS contract procurement processes to date was fair and competitive, and due process was followed in line with Victorian government purchasing guidelines. This conclusion is supported by independent and unqualified probity audit reports.

5.5 Was TPAMS on time and within budget?

In assessing whether TPAMS was on time and within budget, we examined whether milestones were achieved and that costs were within budget.

5.5.1 Milestones

Figure 5C shows the key stages of TPAMS implementation and the originally expected timing of each stage.

FIGURE 5C: TPAMS STAGES

Source: Multimedia Victoria.

At the date of our audit, Stage 1 was completed and Stage 2 was being finalised, slightly behind schedule. Stages 3, 4 and 5 have all commenced and will be ongoing until the contracts are fully implemented.

As previously mentioned, in April 2004, NEC Business Systems was awarded the Victorian office telephony services contract, to operate and update the government’s CBD internal office telephone system. The phased rollout of services to departments and agencies started in August 2004. DOI expects transition to be completed by the end of April 2005.

The telecommunications carriage services tender was issued in September 2004. The first tranche of business (currently worth about $62 million a year) was awarded to Telstra for mobile and data services, and to Optus for voice services.

The 2 tranches were expected by government to reduce departmental costs by a minimum $73 million but, according to the TPAMS project control board, this is likely to be exceeded. Transition for tranche 1 commenced in October 2004 and is expected to be completed in April 2005 for mobile services, and October 2005 for voice and data services.

The LAN transport upgrades project was transferred to user departments. Under the strategy, $10 million was allocated over 2 years (2003-04 and 2004-05) to departments to upgrade their local area networks.

The Network operations Support project developed an audit tool to review the accuracy and reliability of supplier billing processes. The design of this project was completed in May 2004 but was placed on-hold, pending the outcome of the tranche 2 contract award. Once tranche 2 is awarded, the project is planned to progress to implementation. This occurred in early April 2005.

Detailed transition agreements were established with the incumbent suppliers (Telstra and Optus) to ensure business continuity and ease of transition to the new services.

Each stage of TPAMS has more specific milestones, some of which were met and some of which were not. Some milestones were revised. Some milestones cross stages and projects, and this makes it difficult to determine whether specific delays will delay the final deadline. That said, we found no evidence to suggest that the 4 projects will not be fully implemented by the June 2006 deadline.

5.5.2 Budget

In July 2002, the government approved the original TPAMS budget estimate of $55.2 million. This consisted of $34.5 million for capital expenditure and $20.7 million for operating expenditure.

In the TPAMS design phase, Multimedia Victoria decided that building a telecommunication network was too expensive and that services and equipment would instead be purchased by the user departments and agencies. Accordingly, the capital expenditure budget was revised in November 2003.

Figure 5D shows the TPAMS capital and operating expenditure budget, and actual expenditure at 31 March 2005 (the latest available information at the time of preparing this report).

FIGURE 5D: TPAMS CAPITAL AND OPERATING EXPENDITURE BUDGET, AND ACTUAL EXPENDITURE, TO 31 MARCH 2005

(a) The latest available figures available at the time of preparing this report were at 31 March 2005.

Source: Source: Multimedia Victoria.

As Figure 5D shows, 2003-04 expenditure was $10.9 million (compared to the budgeted $20.5 million). The ‘underspend’ includes $4.4 million transferred to departments for their local area network upgrades in 2004-05. Capital expenditure for 2004-05 to 31 March 2005 was $7.4 million, including the $4.4 million transferred to other departments in 2003-04.

The TPAMS governance committee³ which has responsibility for monitoring the progress of TPAMS implementation (which we comment upon earlier in this report), only monitors capital expenditure on a total project basis and does not monitor individual project expenditure or the operating costs administered by the CTO ($20.7 million).

5.5.3 Conclusion

Although there were minor delays and cost revisions, the 4 TPAMS projects are on track to meet the June 2006 deadline, and TPAMS expenditure is currently within budget.

Project cost and milestone reporting to the TPAMS governance committee needs to be improved at the individual project level and for operational expenditure.

Recommendation

3. That the governance committee monitors the capital and operating costs of TPAMS projects at the individual project level and for operational expenditure.

RESPONSE provided by Secretary, Department of Infrastructure

Recommendation agreed. The department recognises the importance of the continued identification and management of emerging risks as a project moves from its procurement phase through to transition and implementation. A review of governance is being undertaken to address the on-going management of TPAMS and other whole-of-government ICT projects. The Auditor-General’s recommendation will be considered as part of that review.

5.6 Were the savings and benefits expected of TPAMS realised?

In assessing whether the savings and benefits expected of TPAMS were realised, we examined if savings and benefits were clearly detailed in the business case and strategy rationale, and if they have been achieved or are indeed achievable.

In 2002, the government approved the business case for the TPAMS projects. The business case anticipated that the TPAMS projects would result in all departments (including the Office of the Chief Commissioner of Police and VicRoads) - and any other agency wanting to participate - being connected to the one telecommunications system. The business case predicted the benefits and savings would enable:

• convergence of voice and data services

• greater competition by suppliers

• greater efficiency in the use of telephony services

• the ability to handle increased volume

• cheaper call costs.

Figure 5E shows the expected benefits of the 3 service provision projects, as stated in the business cases.

FIGURE 5E: EXPECTED BENEFITS OF TPAMS PROJECTS

Source: Multimedia Victoria – TPAMS business case, 2002.

The LAN transport upgrades project already improved access to telecommunications services in regional locations and areas where telecommunication services experienced problems. For example, improvements experienced by the Department of Human Services network include:

• upgrades of facilities at major sites

• replacement of outdated technology at regional locations

• standardised local area networks at regional locations that support the data demands of the department’s main computer applications

• upgrades at 62 metropolitan and rural locations.

There was a high level of competition for both the Victorian office telephony services project and the telecommunications carriage services project tenders. TPAMS contract prices are lower than the pre-TPAMS benchmark tariffs for voice and mobile services, and VicOne tariffs for data services⁴. We expect that telecommunications costs will be lower under these projects.

At the time of our audit, it was too early to assess whether voice and data services have converged, as planned in the business case.

The project control board (which monitors and reports on the progress of TPAMS projects – refer to earlier comments on the project governance structure) told us that they expect the planned savings of $73 million over 5 years to be exceeded. The department expects these savings to be reinvested in more bandwidth and other telecommunications services. There will not be real dollar savings, but a greatly improved service. This improved service will have an implementation cost of $55.2 million.

Conclusion

It is too early for us to determine if the expected benefits and cost savings from TPAMS will be achieved. However, early indications are that telecommunication prices will be lower under the new contracts, and that regional areas will have better telecommunications services.

5.7 Were the projects’ risks adequately managed?

We examined if adequate processes were followed to identify key project risks and to manage these risks. Specifically, we examined measures to manage the projects’ financial risks (particularly project costs and expected savings), and stakeholders’ expectations of quality, timeliness and the degree to which projects would meet user requirements.

Key risks identified for TPAMS project before the tendering process were:

• departments or agencies opting out of the memoranda of understanding and the level of telecommunications carriage services for which they committed

• problems associated with phasing in the new contracts

• failure to meet the June 2006 deadline for all projects

• litigation and lobbying by unsuccessful tenderers.

As the projects progressed, the following further risks emerged:

• inadequate or unclear stakeholder communications

• that departments would not meet their contractual obligations

• contract objectives that were unclear or misinterpreted

• disputed contract deliverables or outputs

• inadequate information or data in contracts

• unclear requirements by departments

• conflicts of interests

• vague or ambiguous contracts.

We found that there were adequate governance and risk mitigation processes to ensure that all risks were monitored and adequately reviewed. Some of the early risks identified were solved by:

• suppliers agreeing to limit the government’s exposure to liability

• a management process put in place to monitor non-performance

• allocating specific risks to relevant parties

• establishing a process that monitored the costs of risk management.

Conclusion

The TPAMS projects’ risk management and monitoring mechanisms are robust. However, as programs move into later stages, risks will change. We expect 2 risks to increase and they will need to be adequately managed, namely the department and agency take-up of telecommunication services and the ability of the CTO to manage a variety of suppliers of telecommunications services.

Recommendation

4. That the CTO ensures it is able to identify and manage emerging risks as TPAMS projects move into later stages.

RESPONSE provided by Secretary, Department of Infrastructure

Recommendation agreed.

¹ The TPAMS governance committee comprises 5 members, including one DOI representative, one stakeholder representative and 3 external members with experience in government and the telecommunications industry.

² The balanced scorecard is a methodology where performance is judged on cost, quality and impact on service facilities, staff and to the users of the services.

³ The TPAMS governance committee comprises 5 members, including one DOI representative, one stakeholder representative and 3 external members with experience in government and the telecommunications industry.

⁴ The benchmark prices were identified in the Commonwealth/States Benchmarking Report, September 2004, prepared by Gibsonquai and AAS (a private sector consulting firm).