/ County Government Saves $700,000 Annually, Achieves Lean IT with Private Cloud
Overview
Country or Region:United States
Industry:Government—Local
Customer Profile
King County provides services to more than 1.9 million people in the Puget Sound area of Washington State. It employs more than 13,000 people in more than 21lines of business.
Business Situation
King County wanted to replace a decentralized IT infrastructure with a consolidated datacenter and use private cloud computing to more efficiently deliver IT services to county departments.
Solution
It deployed a standardized virtualized environment on the Windows Server operating system and Hyper-V virtualization technology, and manages an infrastructure-as-a-service (IaaS) private cloud with Microsoft System Center 2012.
Benefits
Reduced costs
Improved utilization
Elastic scalability
Improved productivity and innovation
Empowered IT staff, better countywide service / “Our staff needs an agile, elastic, and scalable infrastructure that delivers more automation and self-service. So we built them an IaaS private cloud running on Windows Server and Hyper-V.”
Bill Kehoe, Chief Executive Officer, King County
King County wanted to achieve lean government operations and provide better service for its 1.9 million constituents. The King County IT department (KCIT) consolidated 28 data closets into one state-of-the-art datacenter and chose Windows Server with Hyper-V virtualization technology and Microsoft System Center datacenter solutions to build a virtualized environment and a private cloud to provide cost-effective infrastructure-as-a-service (IaaS)capabilitiesto departments. KCIT now provisions a virtual machine in two hours, compared to three months for a physical server, and it costs 40 percent less. King County reduced planned downtime by 90 percent, infrastructure costs by US$700,000 annually, and storage costs by 30 percent. And by delivering responsive IT services, KCIT enables efficient government processes that translate into better value for taxpayers’ dollars.

Situation

Located in the state of Washington and encompassing the city of Seattle, King County has more than 1.9 people living in an area of 2,134 square miles. It provides fiscally responsible, quality-driven, regional services for healthy, safe, and vibrant communities. Behind the scenes, the King County Department of Information Technology (KCIT) works hard to ensure the government runs as efficiently as possible on a cost-effective infrastructure. “Although our 13,000 employees work in different lines of business across more than 253 offices and facilities, we share a common goal: to provide excellent service at minimal cost to our citizens,” says Bill Kehoe, Chief Information Officer at King County. “In IT, we do everything we can to enable this goal.”

The county’s many services depend on more than 1,600 unique IT solutions and applications that were housed on aging mainframes and servers running disparate operating systems and spread among 28 datacenters across the county. For example, the Facilities Management Division was facing server room improvement costs of US$1.6 million. “Think about the number of staff hours necessary to support hundreds of unique applications and manage disparate servers and mainframes,” says Kehoe. “With a cost-conscious mindset in KCIT, we wanted to find a better way to enable consistent, efficient, and standardized IT services to our internal customers behind the firewall—and save taxpayers money.”

While the county had a few isolated virtualized environments, employing both VMware and Microsoft hypervisors, King County had not embraced virtualization on a large scale. Consequently, provisioning new computing resources was a manual process. It could take three months to fulfill requests for new servers. This significantly slowed the test development cycle and extended time-to-benefit for new applications and services. “We received, on average, 100 requests for new servers a year,” says Bob Micielli, Manager of Enterprise Services at King County. “That adds up to a lot of time waiting for infrastructure—9,000 days to be exact. During that time, people couldn’t make progress with their projects.”

“Much like our counterparts in the private sector, we want to provide modernized technology platforms for alternative service delivery, such as increased use of services over the Internet and in the cloud,” says Kehoe. “But first, we had to consolidate our infrastructure. Then, we had to choose the right virtualization technology to build a private cloud and begin our journey toward delivering county services to residents with greater speed and efficiency.”

Solution

In 2011, King County began a major reconsolidation project, eventually moving more than 700 servers into a state-of-the-art datacenter operated by Sabey in Seattle. When this project was completed in early 2012, KCIT decided to align its virtualization initiatives solely with Windows Server and Hyper-V virtualization technology and not VMware.

“The Microsoft cloud operating system vision resonated with what we wanted to accomplish in our new datacenter: stand up a large virtualized environment and build a private IaaS [infrastructure-as-a-service] cloud,” says Kehoe. “Windows Server, Hyper-V, and System Center provide us with all the tools that we need to develop a cost-effective, highly secure multitenant environment to efficiently deliver IT services to our customers. Also, Microsoft licensing for Windows Server and Hyper-V was 30 percent less expensive than the VMware option.”

A Standardized Virtualized Environment

In September 2012, KCIT created its standardized virtualized environment (SVE) running on the Windows Server 2008 R2 operating system with Hyper-V technology and Microsoft System Center Virtual Machine Manager 2008. The new datacenter architecture also incorporates a NetApp storage management solution and the Cisco Unified Computing System. The SVE consists of 11 hosts running 240 virtual machines. While there are still some VMware servers in the Sabey datacenter, KCIT will be migrating all of them to the SVE by the end of 2015.

Private Cloud to Support New IT Service Model

With the release of the Windows Server 2012 operating system and System Center 2012 Service Pack 1 (SP1) datacenter solutions, KCIT had everything it needed to launch its private cloud and start offering IaaS to internal customers behind the firewall. “With this version of Hyper-V, Microsoft made a tremendous leap in technology when it comes to reliability, resiliency, and capacity. For example, you can allocate 64 virtual processors per virtual machine,” says Bob Micielli, Manager of Enterprise Services at King County.

In February 2013, KCIT began providing point-and-click resource provisioning and reliable, scalable, on-demand compute, storage, and networking resources for its internal customers. It created a catalogue containing 17 end-user services. One of them is called the Server Storage and Database. Departments across King County order services through the Server Storage and Database team. KCIT defined service-level agreements (SLAs) and performance metrics for each service and is accountable for achieving them. Using Hyper-V live migration and storage live migration to move multiple, running virtual machines and virtual hard disks between physical hosts or storage locations with no downtime helps to ensure that KCIT achieves its SLAs.

As a county government, King County must comply with the strict tenants of the Health Insurance Portability and Accountability Act and the Criminal Justice Information System. “We can assure our health and criminal justice departments that our multitenant environment is secure and that patient information and police records would not be accessible by any other departments,” says Micielli. “With Hyper-V, you can logically separate the data to avoid co-mingling of information. We also used Hyper-V Network Virtualization to further isolate tenants’ virtual environments.”

Integrated Management Tools

KCIT is using System Center 2012 SP1 to manage both virtual and physical machines in its private cloud, including the Operations Manager component to monitor the entire SVE infrastructure of hardware, operating system, and applications. IT staffers use the Virtual Machine Manager component of System Center to provision virtual machines using templates. Requests are initiated by customers through a custom Microsoft SharePoint form, which immediately creates a Service Desk ticket and emails the SVE team with the request. Information about the virtual machine request including contact information, budget information, and virtual machine sizing is collected through automated Windows Management Instrumentation scripts, and customers are billed quarterly based on the collected information.

KCIT is using multitenancy capabilities within Virtual Machine Manager to create separate roles for IT staff across King County according to job function and department, thereby ensuring that no one gains access to servers or data outside of their jurisdiction. “By implementing the Microsoft cloud operating systemvision, we have everything we need to provide a secure, multitenant environment for our customers, all managed through a single pane of glass,” says Micielli.

“Customers save money by accessing a large pool of virtual resources available in our datacenter, so they no longer have to worry about maintaining their own infrastructure,” says Kehoe. “We can show customers, using quantifiable data, the savings they would get from running their applications and storing their data in our private cloud, rather than on their own physical hardware. The Facilities Management Division saved $1.6 million by running its services in our IaaS private cloud.”

Next Steps

KCIT has built a cluster on the Windows Server 2012 R2 operating system and will migrate all servers in the SVE to this version by the end of March 2014. “We’ve already seen faster live migration technology with Windows Server 2012 R2 running in a failover cluster,” says Micielli.

KCIT was already using dynamic disks, a feature within Windows Server that enables multiple hard disks within a computer to manage data and increase performance and reliability. However, IT staffers are looking forward to improving storage performance and flexibility through storage tiering, which automatically moves frequently accessed data to faster (solid-state drive) storage and infrequently accessed data to slower (hard disk) storage. “The latest storage technologies in Windows Server 2012 R2 offer a Server Message Block-based storage solution that uses the File and Storage Services server role. We expect to reduce storage costs even more than we already have,” adds Micielli.

The second phase of the SVE running on Windows Server 2012 R2 will focus on automation and self-provisioning. “Now that we have a mature platform in place, our plan is to start using some of the advanced, next-generation featuresthat Windows Server 2012 R2 has to offer,” says Micielli. “This will further enhance our provisioning methods and continue to improve IT staff productivity. During the second quarter of 2014, we’ll be using System Center 2012 R2 for workflow orchestration, self-provisioning, and to roll out multitenancy to more county departments.”

Benefits

King County is relying on Windows Server, Hyper-V, and System Center to achieve its vision for a well-managed datacenter and an IaaS private cloud that delivers expedient, reliable services to its departments. “Our staff needs an agile, elastic, and scalable infrastructure that delivers more automation and self-service,” says Kehoe. “So we built them an IaaS private cloud running on Windows Server and Hyper-V.”

Reduced Costs

Since King County consolidated its datacenters and standardized on the Windows operating system, it reduced its total cost of ownership by $700,000 annually. KCIT reduced storage costs by approximately 30 percent using dynamic disks to allocate only the required amount of storage for each virtual machine.

Before the datacenter consolidation project and the IaaS private cloud solution, some of the smaller departments at King County couldn’t afford to ask for a test environment, let alone run a new service in production. “Now we can provide a virtual server at about 40 percent of the cost of a physical server, so all our business users can afford additional computing resources to provide new services,” says Micielli.

Improved Utilization

Compared to a physical infrastructure, King County has significantly reduced the physical footprint of its datacenter. Over the past year, KCIT has virtualized more than 240 servers. Those physical servers would have occupied 16 cabinets. Today, with a density of 25 to 35 virtual machines per Hyper-V host and eight hosts fitting into a single 10U blade chassis, the new standardized virtualized environment only occupies six-tenths of a single cabinet.

“And due to dynamic disks and NetApp thin provisioning, KCIT is using 33 percent of the traditional storage footprint you get with environments that use fixed provisioned disks,” says Micielli.

Improved utilization freed up 15 cabinets that can be repurposed for the Regional Datacenter Co-Location and Managed System Services—a brand new, revenue-generating King County service made possible by the high density achieved with the SVE at Sabey. “We can provide excellent service to businesses in the county at a relatively low cost,” says Kehoe. “This new service extends the value of our datacenter and improves our return on investment.”

Elastic Scalability

With the SVE, King County gets the benefit of scalable resource allocation to accommodate fluctuating demands from some of its customers. This obviates the need for standing up extra hardware for just-in-case scenarios.

During 2013 elections, the elasticity of cloud computing demonstrated the system’s agility by responding to a surge of hits on the website from people tracking election results. “In the old days, we would have set up a physical environment to support a maximum amount of users at a huge up-front investment that goes largely unused,” says Micielli. “But with our SVE, when we saw 164,000 hits on the morning of the election, it took only a few clicks to add additional CPUs to the virtual machines to accommodate the extra traffic, and the site continued to perform well.”

Improved Productivity and Innovation

Staff productivity has increased thanks to automated virtual machine provisioning. It used to take three months to provision a server, now it takes less than two hours to provision a virtual machine. This saves time for IT staff to work on quality control, architecting, and strategic projects. And for the employees across county departments, the vast majority of the 90,000 days of waiting for infrastructure has disappeared. People can get to work on their projects right away, expediting government processes across the county.

Now that requisitioning services through the IaaS private cloud is so much faster, more IT projects are being proposed, increasing the culture of innovation at King County. “In the old days, we may not even have started an IT project because of the time it took to provision servers,” says Micielli. “Today, if someone has a good idea, we can spin up a development environment and help bring that idea to fruition.”

Reduced Planned Downtime

Since the SVE was implemented, employees have rarely been inconvenienced by planned downtime. For maintenance tasks, KCIT staffers now use live migration to move multiple running virtual machines and virtual hard disks between physical hosts or storage locations with no service interruption. “We have reduced the need to plan for system downtime by 90 percent,” says Micielli. “To give a real-world example, the last time we planned downtime was more than seven months ago.”

Empowered IT Staff, Better Countywide Service

Today, it only takes a few hours for KCIT to provision servers and deliver cloud-based infrastructure services to its internal customers. KCIT has become a strategic partner, supporting county business in a timely manner. “With cloud-enabled agility, we can answer our customers’ requests for computing resources faster,” says Micielli. “Then they can turn around and expedite the use of those additional resources to support improved services to the public.”

During a project conducted by the Department of Transportation, KCIT provided extra Windows-based web server capacity within hours, cloning the virtual machine, changing the IP address, and turning over a second server to the application support vendor. When King County wanted to schedule training on SharePoint 2013 in support of the county’s “SharePoint 2013 in the Cloud” project, a day before the training it was discovered that an entire lab environment needed to be built for the class. KCIT quickly provisioned a virtual machine and cloned it into other virtual machines to support the training. And when the Electronic Document Management System needed an additional terabyte of disk capacity quickly for one of its virtual machines, it wasn’t a problem.

“My vision was to turn IT into a service organization to better support the county’s lines of business—health clinics, public parks, court houses, transit stations, recycling depots, water facilities—and to ultimately improve services for the people living in this community,” concludes Kehoe. “Embracing the cloud vision and putting it into practice at King County has taken us a long way toward achieving that goal.”

Transform the datacenter

The hybrid cloud from Microsoft transforms the datacenter by extending existing investments in skills and technology with public cloud services and a common set of management tools. With an on-premises infrastructure connected to the Windows Azure platform, you can deliver services faster and scale up or down quickly to meet changing needs.

For more information about transforming the datacenter, go to:

For More Information
For more information about Microsoft products and services, call the Microsoft Sales Information Center at (800) 426-9400. In Canada, call the Microsoft Canada Information Centre at (877) 568-2495. Customers in the United States and Canada who are deaf or hard-of-hearing can reach Microsoft text telephone (TTY/TDD) services at (800) 892-5234. Outside the 50 United States and Canada, please contact your local Microsoft subsidiary. To access information using the World Wide Web, go to:

For more information about King Countyservices, visit the website at: