As of writing this article, according to the World Health Organization (WHO), there are 85,641 cases of the Novel Coronavirus Disease (COVID-19) in 57 countries which have led to 2,933 deaths. The WHO is recommending that people should:
wash their hand frequently
maintain social distancing
avoid touching eyes, nose and mouth
practice respiratory hygiene
seek medical assistance when needed
The above WHO recommendations are both at the individual and societal responsibility levels that each one of us should follow. In today’s hyper-connected global economy, societal responsibility is of the utmost importance. By all indications, COVID-19 which still does not have a vaccine yet is very close to becoming a pandemic. Organizations around the world have a societal responsibility to also help in containing COVID-19 from spreading. While most organizations around the world do not have the resources to monitor for all the recommendations by the WHO but these organizations still can have an impact on helping maintain social distancing and keeping their employees informed.
In terms of helping maintain social distance, organizations can create possibilities for people to work from home even if they are not sick. The incubation period of COVID-19 is 14 days and by the time it is known that an individual is infected s/he might have unknowingly affected others. By having people work from home, organizations can help slow or even stop the spread of COVID-19 within their own organizations. Unless organizations are in an industry that requires physical labor, most of the people around the world sit in front of a computer screen to do their work which means they can work from home. In order to set up people to work from home, organizations need to discuss the possibilities with their Chief Information Officers (CIOs). If your organization is not large enough to have a technology leader on staff then generally speaking, here is what you need to do:
Create your own, buy or subscribe to a Virtual Private Network (VPN) service
Create your own, buy or subscribe to a Teleconference service
Remind employees to follow all the same rules of information security at home as if they were working from their work locations
Provide laptops, ipads, and phones as needed for their work
Provide paper, pens, pencils and other stationery as needed for their work
Reimburse them for electricity and mobile service charges (if you can not provide phones) to them
Going back to the original question of if technology can help during a pandemic, the answer is a resounding yes! And while you are creating work-from-home possibilities, keep in mind that you do not need an excuse for a global health crisis to help your employees.
This research paper analyzes the Center for Medicare and Medicaid (CMS)’s Healthcare.gov project in detail and makes recommendations on what could have been done differently. The project had 55 federal contractors working on it but this research paper will concentrate on only three. These federal contractors are:
CGI Federal who was developing and implementing the Federally-Funded Exchange (FFE). The estimated value of the contract was $93.7 million and it was awarded in December 2011.
Optum/QSSI was developing the Data Services Hub that would verify citizenship, immigration status and tax information. The estimated value of the contract was $144.6 million and it was awarded in January 2012. Optum/QSSI was also developing the Enterprise Identity Management (EIDM) that would provide enterprise-wide credentials and single sign-on capability. The estimated value of the contract was almost $110 million and it was awarded in June 2012.
Terremark Worldwide, In., (acquired by Verizon) was going to help increase CMS’ Platform-as-a-Service (PaaS) capabilities in the CMS cloud-computing environment. The total estimated value of the contract was $55.4 million and multiple task orders were issued until the summer of 2013.
The following tables summarize this research paper:
Table 1: Key Inputs
Key Inputs of the Project
· Affordable Care Act
· FFE RFP
· Data Services Hub and EDIM RFP
· PaaS RFP
Table 2: Key Components
Key Components of the Project
· Agile Methodology
· Project/System Integrator
· Parallel “stacking” of phases
· CMMI Level 5 Maturity
· Agile Methodology
· CMMI Level 3 Maturity
· Agile Methodology
· Data Services Hub Documents
· EIDM Documents
· Architecture diagram
Table 3: Quality of Project Management – Qualitative View
Qualitative View of Project Management
· Government vs. Private industry projects
· Test plans and test reports
· Requirement changes
· Lessons learned from a state exchange
· Requirement changes
· Previous benchmarking and audits used
· Issue escalation
· Poor coordination
Table 4: Quality of Project Management – Quantitative View
Quantitative View of Project Management
· HHS Enterprise Life Cycle
· Highly metrics-driven
· Use of charts
· Delayed processing of orders
Table 5: Project Management Successes and Failures
Key Successes and Failure of Project Management
· Pressure from White House
· Lack of business processes
· Miscalculated costs
· Various technical options were not considered
· Changing Requirements
· Data Services Hub and EIDM
· Buggy Data Services Hub and EIDM
· Financial Success
· Hardware Outage
· Project Management
Table 6: Lessons Learned
Lessons Learned in Project Management Best Practices
· Roles and Responsibilities Matrix
· Full-Time Project Manager
· Business Processes and Governance
· Requirements Management
· Communications and Sharing
· Metrics and Measurements
· Methodologies and Documentation
Table 7: Recommendations
· Project Manager
· Cross-Functional Team
· Team Satisfaction Survey
· Pilot Approach
The Affordable Care Act (ACA) is the nation’s healthcare reform law enacted on March 23rd, 2010. Under the law, a new “Patient’s Bill of Rights” gives the American people the stability and flexibility they need to make informed choices about their health. There were numerous reasons why healthcare reform was critically needed in the United States, including:
High health insurance rates and lack of coverage by many: In 2013 the Congressional Budget Office (CBO) estimated that 57 million Americans under the age of 65 are uninsured; representing 1 out of 5 in the population.
Unsustainable healthcare spending: Healthcare spending represented 17.9% of the nation’s Gross Domestic Product (GDP) in 2011.
Lack of emphasis on prevention: 75% of healthcare dollars are spent on treating preventable diseases, however only 3 cents of each healthcare dollar goes toward prevention.
Healthcare disparities: Healthcare inequalities related to income and access to coverage exists across demographic and racial lines.
On October 1st, 2013 Healthcare.gov went live as part of the technical implementation of the ACA reform to help Americans buy healthcare insurance, however, the release was an astronomical failure. The cause and contributing factors that led to issues with this project are explored in detail. Focus is placed on looking at CMS’ capabilities from a Project Integration and Project Management perspective. Additionally, our analysis will assess the role of the major Federal contractors in the project. Examples are included to show how contributing factors such as scope creep, schedule constraints and lack of adequate testing led to a website that provided an inadequate customer experience.
This research paper provides a descriptive review and analysis of the Healthcare.gov project. During our analysis, we used Kathy Schwalbe’s (2014) Three-Sphere Model for System Management which entails organizational, business and technological perspectives of Project Management. We utilize these perspectives to determine what went wrong with the Healthcare.gov project from the Federal Government, CGI Federal (contractor), United Healthcare QSSI (contractor) and Verizon (contractor) points of view. Furthermore, building on these unique perspectives, we analyzed the stated objectives and real implications of the project, the quality of the project management from a qualitative and quantitative perspective, key successes and failures factors, key lessons learned project management best practices, and recommendations for what might have been done differently.
In order to set the context of the research paper, we have to understand what the CMS does, what the reason for the Healthcare.gov website is, and why the on-time completion of the website was a priority. Additionally, we will look at the key inputs, components, and deliverables of this project.
3.1 About CMS
According to the www.CMS.gov and www.hhs.gov/about/foa/opdivs/index.html websites, CMS is one of the operational divisions of the Department of Human and Health Services (HHS). CMS is responsible for providing oversight on the Medicare and Medicaid program, Health Insurance Marketplace and related quality assurance activities. It has 10 regional offices with its headquarters in Maryland. The Administrator for the CMS is nominated by the US President and confirmed by the Senate. Figure 1 (Priorities, 2014) below shows CMS’s 2013 budget that accounted for 22% of the entire US Federal Government Budget.
Figure 1: Federal Budget Distribution
3.2 Why CMS was given the project?
CMS was designated as the chosen one by the Obama administration for obvious reasons. As discussed earlier the purpose of the ACA was to enable all Americans with the ability to buy health insurance. We also defined the CMS as an organization that provides healthcare to the elderly, disabled, and those who are not financially capable of buying healthcare. Healhcare.gov project began as a sub-agency under HHS called the Center for Consumer Information and Insurance Oversight (CCIIO). This office’s charter was to support a successful rollout of the ACA. In 2011, the secretary of HHS, Kathleen Sebelius “Citing efficiency gains…” stated that the CCIIO would be moved under CMS (Reichard, John. Washington Health Policy Week in Review Sebelius Shuffles Insurance Oversight Office into CMS, Shifts CLASS Act to Administration on Aging. January 10, 2011). The Obama administration insisted this was a way to control IT costs and leverage economies of scale through existing investments and infrastructure. The Republican opposition believed this was another example of “…resources being diverted from seniors’ health care to be used to advance the Democrats’ new government-run health care entitlement” (Reichard, John. Washington Health Policy Week in Review Sebelius Shuffles Insurance Oversight Office into CMS, Shifts CLASS Act to Administration on Aging. January 10, 2011).
3.3 About the Federal Contractors and their relationship with the CMS
3.3.1 CGI Federal
CGI Group Inc., a Canadian company acquired American Management Systems (AMS) in 2004 to enter the U.S. Federal Government market. Due to this acquisition of an American Federal contractor by a foreign entity, there was a “firewall” created so that CGI Federal (formerly called AMS) could continue to work on Federal contracts. This “firewall” entailed that CGI Federal would not share Federal client information with CGI Group Inc. thus CGI Federal became a wholly-owned subsidiary of CGI Group Inc. This acquisition proved to be very lucrative for CGI Group Inc. CGI Federal became one of the most profitable business units due to the Healthcare and Human Services division. This division provides IT services in the areas of provider-based services, public health surveillance, portal integration, security, enterprise architecture, service-orientated architecture, business intelligence, and application development.
In September 2007, CGI Federal, among 16 other Federal contractors was awarded the Enterprise Systems Development (ESD) Indefinite Delivery Indefinite Quantity (IDIQ) contract by the CMS. The purpose of the ESD IDIQ was to support CMS’ Integrated IT Investment & Systems Life Cycle Framework and various IT modernization programs and initiatives. Although there were no task orders issued under this contract at the time it kept the door open for future task orders to the 16 contractors.
The Healthcare.gov project was competitively bid under the ESD IDIQ. This bid resulted in four contractors becoming the finalists. Out of those four contractors, CGI Federal was selected on September 2011 as the awardee based on “best value”.
Founded in 1997, Quality Software Services Inc. (QSSI) is an established Capability Maturity Model Integration (CMMI) Level 3 organization with a proven track record of delivering a broad range of solutions with expertise in Health IT, Software Engineering, and Security & Privacy. Based in Columbia, MD Optum/QSSI is a subsidiary of UnitedHealth’s Optum division and was acquired by Optum in 2012.
Optum/QSSI is privately held with about 400 employees and collaborates with both the public sector and the private sector to maximize performance and create sustainable value for its customers. In its 15 year existence, the company has cultivated a process-driven, client-focused method of IT solution development, which, has solidified its reputation as a capable IT partner in both the federal and commercial marketplace. In the federal landscape, Optum/QSSI has established itself as an industry in the field of Health IT and this reputation was key in the company’s selection as a federal contractor for Healthcare.gov.
Terremark is now a Verizon company, dedicated to combining the strongest cloud-based platform with the security and professional services that are necessary to conduct today’s enterprise and public sector business across the next-generation IT infrastructure. At the center of Verizon’s capabilities is its enterprise-class IT platform, which combines advanced IT infrastructure with world-class provisioning and automation capabilities which is what CMS leveraged in this case. Verizon’s standards-based approach fully aligns with today’s enterprise business requirements driven by agility, productivity, and competitive advantage.
Verizon Terremark was a natural fit at the CMS through a long-standing relationship. Verizon manages and maintains the entire HHS Wide Area Network (WAN) along with ancillary services such as security services, mobile solutions, and unified communications. Verizon also developed a homegrown fraud detection service is used to identify toll-free fraud to pursue Medicare and Medicaid fraud saving the agency millions.
3.4 Key Inputs of CMS
For the Healthcare.gov project, we identified the following key inputs for CMS:
Patient Protection and Affordable Care Act (ACA): The ACA became law on March 23rd, 2010 under the Obama Administration. The legislation was passed to address various consumer health insurance purchase issues such as a denial of coverage due to pre-existing conditions, stopping of coverage if patients became sick, lifetime benefit limits and access to affordable healthcare. The ACA mandated the creation of “exchanges” that would be used by consumers to compare and buy a Qualified Health Plan (QHP) based on their state of residence, income level, age, and other factors. These exchanges could be created at the state level or at the federal level. If states decide not to create their own exchanges then they would have the option to redirect their constituents to the federal exchange to buy healthcare insurance.
States: The various states and Washington D.C. informed CMS if they intend to create their own exchanges or they would utilize the exchange developed by the Federal government. States also had the flexibility to utilize the federal exchange when they wanted and thus initially 26 states opted to have their constituents go to the federal exchange to purchase healthcare insurance.
People/Team: CMS assigned a part-time Project Manager to the Healthcare.gov project.
3.5 Key Inputs of Healthcare.gov for Federal Contractors
The Healthcare.gov project is one of the most complex systems in recent times. The project entailed 55 contractors working on various aspects of the system. These contractors were responsible for the creation of a robust network/infrastructure, the development of a website front-end, the Federally-Funded Exchange (FFE), the Data Hub and the EIDM. Additionally, this system receives eligibility and verification information from various other federal government agencies as the consumer fills out the online form. The following figure (Ariana Cha, 2013) below shows the complexity of the information flow of the entire system.
Figure 2: Healthcare.gov contractors and agencies processes
3.5.1 CGI Federal
CGI Federal was one of the prime contractors for the Healthcare.gov project. It had the following key inputs:
Request for Proposal (RFP): An RFP was one of the first key inputs for the Healthcare.gov project. It required the establishment of a FFE that would be used for eligibility and enrollment, plan management, financial management, oversight, communication, and customer service. The following Figure (Desai, 2013) shows the FFE Concept of Operations:
Figure 3: FFE Concept of Operations
Requirements: After the contract was awarded, first the CCIIO and then various other representatives within CMS provided requirements to CGI Federal for the FFE. These representatives came from policy, legal and Office of Information Services (OIS).
Request for Proposal (RFP): This was the primary input for the project. It required the development of a data services hub for information exchange and the EIDM for user account registration.
Request for Proposal (RFP): One of the first RFPs released in 2010 was for the infrastructure and essentially Platform as a Service (PaaS) for the ACA as dictated in the 2010 RFP. A Cloud Solutions Executive for Verizon Terremark said Verizon received an award prior to the additional contractors involved. Following the award, CGI Federal and others were asked to develop the system to conform to the Verizon environment.
3.6 Key Components of Healthcare.gov for CMS
Systems Development Methodology: A presentation from April 2012 by CMS’ OIS shows that an “Agile” methodology was used for the Healthcare.gov as shown in the following figure (Services, 2012).
Figure 4: CMS Agile Methodology
Project/System Integrator: CMS took on the role of “system integrator” to manage all 55 contractors.
Implementation Consideration: A McKinsey report shows the parallel “stacking” of all phases for this project as shown below (CMS, Red Team Discussion Document, 2013):
Figure 5: McKinsey Report for CMS
3.7 Key Components of Healthcare.gov for Federal Contractors
3.7.1 CGI Federal
Process Methodology: Patrick Thibodeau indicates in a Computerworld article that CGI Federal attained Capability Maturity Model Integration (CMMI) Level 5 maturity making it only the 10th company in the US to achieve this level. By extension, we can assume that CGI Federal brought the best practices of CMMI for the Healthcare.gov project.
Systems Development Methodology: Based on Federal contracting experience, the Federal contractor would either have their own development methodology or they would use the development methodology of the client (CMS in this case). Research indicates that CGI Federal used an Agile methodology to develop the FFE.
Process Methodology: As a CMMI Level 3 organization, Optum/QSSI has a reputation for process-driven and client-focused methods of IT solutions development. Based on our research it is evident that the company implemented CMMI best practices on the project.
Systems Development Methodology: According to a senior analytics consultant with a major health provider who worked on the project, Optum/QSSI used an agile development methodology similar to the one depicted below (Group, 2014) based on iterative and incremental development with continuous visibility and opportunity for feedback from CMS.
Figure 6: Agile Methodology
Requirements Documentation for Data Services Hub: The Data Services Hub is a central function of the federal exchange which connects and routes information among trusted data sources including Treasury, Equifax, Social Security Administration (SSA), etc. Inputs from CMS which changed in late September 2013 to allow for an account creation before shopping for health plans.
Requirements Documentation for EIDM: EIDM enables healthcare providers to have the ability to use one credential to access multiple applications, serving the identity management needs of new and legacy systems. Inputs from CMS which changed in late September 2013 to allow for an account creation before shopping for health plans.
System Architecture Design: There were an architecture diagram and overall design for the entire system that lost effectiveness due to a lack of accountability to ensure each component was delivered.
Security: Security was a huge component within the requirements for infrastructure, and Verizon Terremark offered a highly secure architecture designed to meet all of the critical compliance and certification requirements. Verizon had been audited against FISMA to the moderate-level and NIST 800-53 for federal customers. Verizon was also asked for advanced security options on the platform such as intrusion detection/intrusion prevention (IDS/IPS), log aggregation, and security event management.
3.8 Key Deliverables for CMS
Website: A website that should be able to provide residents the ability to compare QHPs.
Exchange: A website that should enroll residents by verifying their eligibility based on income level, age, and other factors.
3.9 Key Deliverables for Federal Contractors
3.9.1 CGI Federal
FFE: A fully functional FFE would be ready to go live by October 1st, 2013. The FFE would be the backbone of Healthcare.gov and would seamlessly integrate with the website, the data hub, and the EIDM.
Data Services Hub: This system of Healthcare.gov determines eligibility for financial help. It sends customer data to various government agencies (VA, DHS, Treasury, etc.) to verify eligibility.
EIDM (Proof of Identity): Upon account creation, this system verifies identity with Experian. The system also enables healthcare providers to have the ability to use one credential to access multiple applications, serving the identity management needs of new and legacy systems.
PaaS: Fully operational infrastructure which provides servers and hosting for the Healthcare.gov exchange.
Environmental: Supports power, connectivity, and memory requirements for the environment.
Service Level Agreement (SLA): Rolling out the infrastructure in a timely fashion, offering and executing upon SLA’s required by the Government among other things.
4.1 Project Management Quality of CMS
Quality Planning: Quality planning for government releases is at a different scale than quality planning for private companies. Many factors come into play such as redistributing of the resources through regulation, subsidization, and procurement. As part of CMS’ quality planning phase, the main scope aspects were functionality, features, and system outputs. However, performance, reliability, and maintainability suffered heavily due to time constraints as October 1st, 2013 was the hard deadline.
Quality Assurance: CMS used test plans and test reports to ensure quality coverage were being met as per the requirements. The front-end web interface was indeed completed in time. However, identifying quality system integration was difficult due to the complexity of the back-end sub-systems.
Quality Monitor and Control: During the implementation phase, CMS didn’t take proactive measures to address the issues they found one week before launch. Specifically, when the testers reported server crashes at a scale of 10,000 concurrent users. Additionally, CGI Federal had reported more testing is required yet it appeared that CMS was insensitive to their recommendations. Status reports were supposed to be read, understood and acted upon. HHS followed the Enterprise Life Cycle and CMS was supposed to follow these guidelines.
4.2 Project Management Quality for Federal Contractors
4.2.1 Project Management Quality of CGI Federal
Quality Planning: As a CMMI Level 5 organization CGI Federal had optimized quality processes to deliver appropriate outcomes for the FFE. However, requirement changes seem to be one of the main issues with the project. Requirements were still being revised until the summer of 2013 and kept on evolving even a week before go-live. Additionally, the number of states that were going to join FFE increased from 26 to 34 which created another level of complexity in terms of maintaining the quality on the project.
Quality Assurance: According to Cheryl Campbell, Senior Vice President at CGI Federal, in the Congressional hearings CGI Federal developed the FFE as per the contract requirements. It is interesting to note that CGI Federal was also one of the companies that developed the Massachusetts Health Exchange that was used as a model for the FFE. Hence, we can make the assumption that quality lessons were learned from that project which could have been used for the FFE.
Quality Monitor and Control: CGI Federal is a highly metrics-driven organization. Each project is monitored and measured according to industry “best practices” and proprietary methodologies. Projects are evaluated based on scope, cost, schedule, and other factors to check the health of the project and verify if they are keeping the customer satisfied. But if the requirements continue to evolve then even the best methodologies and measurements are not a match for customers changing their minds.
4.2.2 Project Management Quality of Optum/QSSI
Quality Planning: As a CMMI Level 3 organization Optum/QSSI had a planned quality process to deliver appropriate outcomes for the Data Services Hub and EIDM project deliverables. However, changing project requirements from CMS severely impacted quality planning efforts. For instance, the late requirement change in September that required consumers to create user accounts before browsing the exchange marketplace resulted in higher than expected simultaneous system usage and as a result impacted the functionalities of the EIDM tool which was originally designed to allow consumers to first access the systems, browse the marketplace, and if they wanted a product, create an account. Because the EIDM is only one tool for the federal marketplace registration system, this late requirement change made it impossible to coordinate and plan quality processes with other contractors who worked on portions of the registration system to ensure the appropriate performance outcomes before the go-live date of October, 1st.
Quality Assurance: Both the Data Services Hub and EIDM deliverables met quality assurance satisfying CMS’ requirements and all relevant quality standards for the project according to Andrew Slavitt’s, Group Executive Vice President at Optum/QSSI, during his Congressional hearings. It is important to also note that Optum/QSSI developed an EIDM tool for two other CMS systems. This EIDM tool followed benchmarking and quality audits taken from those existing EIDM solutions at CMS.
Quality Monitor and Control: Requirements changes greatly impacted quality monitoring and control. Although Optum/QSSI used quality control tools such as charts to guide acceptance decisions, rework, and process adjustments, changing requirements severely impacted these controls. These changes introduced time constraint challenges and limited system-wide testing, and most importantly user acceptance testing.
4.2.3 Project Management Quality of Verizon
Extensive delays in processing orders for additional capacity, provisioning resources, and implementation also caused Verizon a lot of angst with the CMS customer.
Management within Verizon also failed to run some of the concerns up the executive flagpole to make leadership aware of issues that could have prevented delays or numerous escalations by CMS.
Verizon’s project management failed on many accounts. Poor coordination was to blame between multiple project managers assigned to the project within.
4.3 Project Management Key Successes and Failures
If we review the congressional hearings and documentation, they reveal that the Healthcare.gov project was a high priority project for CMS. In conversations with Federal contractors, CMS would start by saying “this is what the White House wants…”. It is still unclear whether this prefix was used because directions were actually coming from the White House or whether it was just used to indicate the importance of the project. Regardless of the intentions, one thing is for sure that they were not followed by action since there was not a dedicated full-time Project Manager to manage the project from kickoff to implementation. Most likely decisions were made by committees as is often the case with large government projects.
A big piece of the Healthcare.gov project included behind the scenes business processes even before the technology was to be considered. These business processes and governance entailed not only coordinating with 26 states but also with insurance companies. The picture below depicts an exhaustive list of stakeholders that were affected by the project:
Figure 7: FFE Stakeholders
From a business standpoint, CMS failed to calculate in advance the true cost of the entire Healthcare.gov project. Additionally, even after reports by McKinsey indicating a danger of not doing end-to-end testing and warnings from CGI Federal in their August 2013 status report (Federal, 2013) that testing could be an issue, CMS ignored these experts and went full steam in going live on October 1st, 2013.
Research indicates that some COTS products and custom software were developed to standup Healthcare.gov. It also seems like CMS failed to look at the various internal and external “firewalls” the Healthcare.gov system needed to pass through.
4.3.2 Federal Contractors
22.214.171.124 CGI Federal
FFE: According to Congressional hearings, the CGI Federal representative indicated that they had provided a fully functioning FFE as per contract requirements by October 1st, 2013. This was their success factor.
Changing Requirements: CGI Federal was responsible for developing the FFE. It was put under the spotlight for not providing recommendations holistically for the entire project. It is evident that requirements were changing and new states were being added. But there was no push back from CGI Federal to indicate that the required changes would result in quality issues on their end that would affect the entire system.
Testing: While the system did work for an initial couple of users but logging delays resulted in poor customer experience. Research indicates that no end-to-end testing was performed to see holistically how the system would work. CGI Federal could have used their vast amount of industry expertise to inform CMS that no end-to-end testing would result in major issues.
Data Services Hub & EIDM: Based on Andrew Slavitt’s Congressional hearings on Healthcare.gov, Optum/QSSI successfully developed and delivered a fully functional Data Services Hub and EIDM tools. For example, according to Slavitt on October 1st the Data Services Hub processed over 175,000 transactions and millions more after the project launched.
Buggy Data Services Hub & EIDM: In the same Congressional hearings, however, Andrew Slavitt acknowledged that the Data Services Hub and EIDM tools, although worked functionally as designed, experienced performance bottlenecks when the project launched because of the late requirements changes requiring consumers to create accounts before browsing the marketplace. This change resulted in higher than expected simultaneous usage of the registration system and the Data Services Hub eligibility verification tool. Slavitt also admitted to the fact that Optum/QSSI identified and fixed bugs in the EIDM tool days after the October, 1st launch date. The release of code that had bugs was a quality failure and contradicts Slavitt’s earlier comments about delivering a fully functional EIDM tool.
Financial Success: The primary success story for Verizon was that financially the company did far better as a result of this project than initially predicted based in large part to the scope creep. Additionally, Verizon specifically was not the cause for delays or outages on day one of the projects and delivered the infrastructure to support the site by launch date. A significant underestimation of capacity would be to blame for the initial failures of Healthcare.gov.
Hardware Outage: subsequent failure which Secretary Sebelius cites in her testimony was a hardware outage unrelated to project management (Krumboltz, Michael. Helathcare.gov suffers outage as Sebelius testifies that it never crashed.).
Project Management: Key project management failures within Verizon were inherent with the ordering, design or engineering phase, and eventually, implementation suffered due to project management inefficiencies. As discussed previously, Verizon had several members of a project management team supporting the CGI Federal relationship, QSSI, and CMS relationships. There should have been one central program manager who supported the ACA contract for Verizon. The communication failure through project management teams created bottleneck failures that spread throughout the engineering teams.
4.4 Lessons Learned on Project Management Best Practices
Who’s on First: Even though this project needed input from various internal and external stakeholders, clear roles and responsibilities matrix should have been developed. This matrix should have been used by the contractors to see who is responsible for various activities of this large project and who specifically reach to out to in case they ran into bottlenecks.
Project Manager: CMS did not assign a full-time Project Manager for the Healthcare.gov project. For this large-scale project, it would have been prudent to have a full-time project manager responsible for coordinating various activities internally and across various contractors.
Business Process and Governance: Since this was the first time such an endeavor was taking place with multiple stakeholders, it would have been useful to map the business processes of the future state prior to award. Overall business processes would also support governance structures that would help in checking progress and checking alignment with the stated objectives of the project.
Requirements Management: Ever-changing requirements and a change in strategy can affect projects dramatically. For Healthcare.gov, there should have been some baseline requirements established earlier in the project. The baseline requirements would entail the basic functionalities needed by CMS. It would have been useful to use a Requirements Traceability Matrix (RTM) that would be available to everyone on the project. The RTM would not only help all stakeholders be informed of what is going but it would keep the entire team honest as well and perhaps identify any issues before they become a problem later.
Communications and Sharing of Information: The way the Healthcare.gov project was set up it seems like the left-hand did not know what the right hand was doing. This can create problems in understanding issues from a holistic perspective and not knowing if there are dependencies that should be coordinated.
Metrics and Measurements: Reasonable metrics should have been created to assess the health of the project. These metrics should measure the stakeholder and team satisfaction at the beginning and during the project in the project life cycle to determine where adjustments need to be made. Based on these metrics, remediation processes should have been set up so that nothing falls through the cracks.
Methodologies and Documentation: Although it seems like CMS was supposed to follow HHS’ Enterprise Life Cycle (ELC) but research indicates that an Agile methodology was used to develop the Healthcare.gov system. This alludes to conflict in terms of what is supposed to be used versus what was actually being used. Additionally, the vendors who were helping CMS came with their own methodologies. It seems like there were too many methodologies but not a consistent alignment of them across the organization so that all teams were on the same page. In this scenario, the advice would be to understand the various methodologies at play and select the most appropriate one. This selection might also entail having some sort of a hybrid methodology that everyone conforms to. Having one methodology would reduce the amount of documentation that needs to be developed thus freeing up resources to work on the actual needs of the project.
5.1 Project Design Recommendations (CMS only)
Project Manager (PM): The Healthcare.gov project had 55 federal contractors working on it at various times. The system components that these federal contractors were developing were dependent on each other. For example, the Healthcare.gov website needed to communicate with EIDM and Data Hub which would communicate with FFE. Due to these complex dependencies, there was a significant amount of communication and coordination that needed to happen on this project. Additionally, someone needed to see how not only if these system components would work with each other but how the thorough testing of these systems would be the difference between a failed project versus a successful project. Despite the complexity of managing such a large group of contractors and understanding the pros and cons, CMS did not have a full-time PM for Healthcare.gov. While the real reasons for this decision are unknown, we can extrapolate from research that lack of an astute full-time PM was one of the major causes of the issues with Healthcare.gov.
We recommend that a full-time PM should have been assigned to Healthcare.gov who had experience in large-scale implementations. This PM would have the authority to push back on unrealistic timelines, have a holistic view of the project and understand that even though individual system components are being developed, there should be time allocated in the project to perform effective end-to-end testing.
Cross-Functional Teams: As discussed earlier, the system components that the federal contractors were developing had many dependencies on each other. Despite these dependencies, no proof has been found that cross-functional teams were established.
We recommend that in designing the project, the development of cross-functional teams should have been given a high priority. These cross-functioning teams would comprise of government and contractors. These teams would not only create synergies among the various people but should be designed in a way where sharing of lessons learned and recommendations are encouraged.
Team Satisfaction Survey (TSS): When a project falls off the track, oftentimes it is because once management pays attention to it, they are at a point of no return. This is what seems to have happened at CMS. By the time people working on the project started showing their concerns that testing could be an issue, CMS either ignored this or that they did not have a choice and thus went ahead to release a buggy version of the system to the masses.
We recommend that a TSS should be incorporated into the project whose purpose would be to ask people at all levels about their concerns and recommendations of the project. The TSS should collect this information at the beginning and periodically during the project. The TSS should also have a mechanism where actions could be taken promptly by management if there are issues that seem to be recurring. The TSS is not a status report but actually a mechanism to check the pulse of the project.
Pilot Approach: The healthcare.gov project used a “big bang” approach to release the software on October 1st, 2013. This approach resulted in overwhelming the system as users who tried to login found that their online forms were either taking too long to verify their information or that simply they were kicked out of the system. Additionally, it is apparent that the federal government did not anticipate the system errors it was going to get when the system went live.
We recommend that a pilot program should have been created for one of the states. This pilot program would be used to see how the system would perform when it goes live and what kind of issues it might have once it is open to the masses. Lessons learned from this pilot program could have been used to provide a better customer experience once the system went live.
To summarize, it should come as no surprise to those familiar with IT projects that most IT projects fail. A recent Gartner user survey showed that, while large IT projects are more likely to fail than small projects, around half of all IT projects fail in part due to unrealistic requirements, technical complexities, integration responsibilities, aggressive schedules, and inadequate testing. Causes that were all related to the Healthcare.gov project which resulted in its failure. As outlined in this case analysis, these fundamental missteps were the contributing factors that led to issues with this project and ultimately its failure. Based on our analysis CMS did not have the Project Integration and Project Management know-how to manage such a major project. The Agency’s assignment of a part-time project manager to the Healthcare.gov project is evident that leadership did not fully understand the magnitude and importance of the project, and what it took to implement IT projects. Based on our recommendations, it is our hope that such project management missteps are avoided for IT project implementations in the future.
Last week, I took my wife to a well-known local clinic since she had the flu that is spreading across the U.S this year. Since I had my notebook, I started to make some observations in regard to the overall process and customer service. In the following paragraphs, I will attempt to make a surface-level current state analysis and propose a future state to be considered.
As we walked into the clinic, the receptionist smiled at us and asked how she could help. We explained to her that my wife probably had the flu that is going around. She asked us to sign-in and gave us paperwork to fill out since it was the first time we came to this clinic. While filling out the paperwork, I overheard another person come into the clinic and the receptionist inquired if they had been to the clinic in the last 4 months and if not then paperwork needs to be filled out (again). After finishing our paperwork, we gave it back to the receptionist and she asked for my wife’s driver’s license. I am assuming that all this information was needed to create a preliminary patient record on the computer.
30 minutes later the nurse called her name and took us to a small room. In this room, the nurse requested my wife to change into a medical gown and gave us some privacy to do so. After 20 minutes the nurse came back and started typing the information in the paperwork into a computer in the room. The computer was placed in a way that the nurse had her back towards my wife. After typing the paperwork information the nurse typed the blood pressure reading, temperature, height, and weight into the computer. During the data entry, I asked the nurse how come my wife still got the flu since she had taken the flu shot a couple of weeks ago. I was explained that the current flu shot only protects against 4 strains of the flu and the flu strain that my wife had is different.
After the nurse left it took about another 15 minutes before the doctor came into the room and looked at the paperwork and inquired about a medication listed that she could not find in the system. This medication is a Tylenol-type medication but since it was not sold in the U.S. there was no record of it and thus we explained to the doctor that the active ingredient was Acetaminophen and the dosage level. After inputting this information into the computer, the doctor recommended a basic test that would be carried out at the facility.
The doctor left and the lab technician came in about 10 minutes later and took samples for the basic test. Another 20 minutes go by and then the doctor comes into the small room and informs us that nothing serious was found during the test and prescribes antibiotics. The doctor also informs us that she will conduct further tests whose results will be known in the next 48 hours and regardless of the result she will call us. 4 days have gone by and we have not received a call from the doctor.
So that is the current state and it seems pretty typical but now the fun part begins where I propose a few things to think about and how I would hope the process would go in the future…
As we walk into the clinic, the receptionist smiles at us and my wife swipe her health insurance card across a card reader. The card reader signs her in, provides a queue number, the receptionist confirms that the record is accessible and provides a medical gown. After this the receptionist asks if we want to use the self-service kiosk to take readings for blood pressure, temperature, height, and weight or if we prefer that a medical professional take these readings. We prefer the self-service kiosk that takes about five minutes and updates the patient record. The patient record also contains the flu shot information that my wife took several weeks earlier. This information and the readings taken by the self-service kiosk are not displayed anywhere on the self-service kiosk but it gives the option to the patient if they want to print this information for themselves and gives a list of basic tests that are taken. Additionally, she is glad to have access to her patient records available securely online if she wanted to view them herself to see what medical conditions she has had and the insurances she used in the past.
A few minutes go by and then it is her turn. We enter a private room that has a sign outside indicating if the patient is ready or not. The doctor comes in and after some inquiry recommends a basic test that we already know about from the self-service kiosk. Then the doctor leaves and the lab technician comes to take test samples. The test is carried out at the facility while we wait in the private room. Additional minutes go by and the doctor shows up indicating that nothing serious was found and prescribes antibiotics. As the doctor is putting this information into the patient record, it indicates if there is a history of any allergic reactions to this antibiotic and if the current medications will have any effect on my wife. The doctor informs about additional tests that will be conducted and if we request they can provide us a printout about those tests.
As we leave, the doctor indicates that an automated message will be sent to the cell phone to indicate if the results have not found anything and we will have a window of about 5 minutes to call back if we want to discuss something further. In the case the doctor finds something that needs to be explored further, she would call us as soon as possible or within 1 day whichever is sooner after the results have been received.