Instructor: Prof. Michael P. Harris, CCNA CCAIChapter 6
ITSY 2400 – Operating Systems Security Firewalls and Border Security
Operating Systems Security - Chapter 6
Firewalls and Border Security
Chapter Overview
In this chapter, we will learn the basics about the common language of networks, the TCPUDP (OSI Layer 4), and IP(OSI Layer 3)protocols. Learning about these protocols enables us to understand their security vulnerabilities and how these can be mitigated. We will reviewIP addressing, including how it can be used to thwart attacks. You will also be introduced to border and firewall security, which can use characteristics of TCP, UDP, and IP to build more secure networks. Finally, you’ll learn how to configure the firewall capabilities of OS.
Learning Objectives
After reading this chapter and completing the exercises, students will be able to:
Understand how TCP, UDP, and IP work and understand their security vulnerabilities
Explain the use of IP addressing on a network and how it is used for security
Explain border and firewall security
Configure the firewall capabilities in operating systems
Lecture Notes
An Overview of TCP, UDP, and IP
Since its introduction in the early 1970s, Transmission Control Protocol/Internet Protocol, or TCP/IP, has been widely used on networks throughout the world. It is the networking protocol of choice for modern Windows, UNIX/Linux, and NetWare systems. TCP/IP contains nearly 100open protocols that interconnect computer systems efficiently reliably.
The core component protocols within the TCP/IP protocol suite are:
Transmission Control Protocol (TCP)
User Datagram Protocol (UDP)
Internet Protocol (IP)
Understanding Transmission Control Protocol (TCP)
Transmission Control Protocol (TCP)is a transport protocol that maintains communication sessions between software application processes initiated by users on a network. TCP provides for reliable end-to-end delivery of data by monitoring the accurate receipt of packets and by controllingdata flow. TCP accomplishes this by sequencing and acknowledgingpackets, both characteristics that enhance security as part of a connection-oriented servicesapproach to communications.
The TCP frame contains a header and payload data (see Figure 6-1 on page 258 of the text) and is called the TCPsegment. The TCPheader is a minimum of 20 bytes in length and contains the following fields:
Source port Destination port
Sequence number Acknowledgement number
Offset or header length Flag/control
Window Checksum
Urgent pointer Options
Padding
Attackers use this knowledge to scanning networks and launch attacks. TCP UDPport-scanning software may be used to simply collect information about a target, without the target’s knowledge; gain access to a system, or used to crash a system.
An intruder may simply collect open ports/socketsinformation for later use, and not connect. An attacker can use port-scanning software to overrun ports at the target with repeated packets containing the SYN code bit, to establish a communication, and then send repeated RST code bits to prevent immediate responses from the target.
Understanding User Datagram Protocol (UDP)
One limitation of TCP is that its connection-oriented design can create overhead on a busy network. User Datagram Protocol (UDP) can be used as an alternative to TCP for communications that do not require the same level of reliability(handshaking) as provided by TCP. When it transmits data, the TCP/IP suite has the option to transmit data using UDP instead of TCP. UDP employs connectionless services (no handshaking)with no reliability checks such as sequencing and acknowledgements and containing virtually no overhead on top of the IP-baseddatagrams sent. The UDP header has the following fields:
Source port Destination port
Length Checksum
Because it does not use sequencing and acknowledgements, UDP is simpler than TCP and port-scanning attacks are less effective. AUDP port can appear open to a port scanner, when it is really closed. This is because the target may –or may not send back an Internet Control Message Protocol (ICMP) message indicating that the port cannot be reached.
Understanding How the Internet Protocol (IP) Works
Anenterprise may be composed of a series of subnetworks. TheIP-address/subnet mask enables a packet to reach different subnetworks on a LAN and different networks on a WAN.
The Basic Functions of IP
The basic functions of IP are to provide for data transfer, packet addressing, packet routing, fragmentation control, and simple detection of packet errors. Each network host has a
32-bit IP-address(in IPv4), which, when used with its 48-bitMedia Access Control (MAC)address, enables network (LAN) and internetwork (WAN) communications and accurate delivery of packets. The MAC address, sometimes called the Burned-In Address (BIA), is permanently burned into the Network Interface Card (NIC).
IP as a Connectionless Protocol
IP is a connectionless protocol because its primary mission is to provide network-to-network addressing and routing information, and to change the size of packets when the size varies from network to network. When the OSI Layer 4,TCPsegment, is encapsulated with the additional OSI Layer 3,IP header information, the entire unit is called a datagram or packet, as shown in Figure 6-3 on page 264.
The IP packet header consists of the following fields, as shown in Figure 6-4 on page 265:
Version IP header length (IHL)
Type of service (TOS) Length
This list continues on pages 264 through 266 of the text.
How IP Addressing Works
IPaddressing is used to identify a specific host and the network on which it resides. The IPaddress format is called the dotted decimal notation address. It is 32 bits long and contains four octets, which are decimal values representing 8-bit bytes. An IP address in binary looks like this: 10000001.00000101.00001010.01100100. This converts to 129.5.10.100
There are five IP address classes, Class A through Class E, each used with a different type network. Classes A through Care intended for normal unicast addressing, but each class represents a different network size.
Using a Subnet Mask
IPaddresses require a subnet mask. A subnet mask is used for two purposes: to determine how portions of addresses on a network are divided into the network ID and the host ID, and if needed-divide a network into subnetworks to control network traffic.
Creating Subnetworks
To divide the network into subnets, a classlesssubnet maskdetermined by the network administrator, divides a network into subnetworks and valid host ID ranges. Using a subnet mask to divide a network into a series of smaller networks enables routing devices to effectively ignore traditional address class designations (classful), and therefore creates more options for segmenting networks through multiple subnets and additional network addresses, to overcome the classful network size limitation in IPv4. A newer way to specify the classlessaddressing (subnetting)subnet mask is by using theClassless Inter-Domain Routing (CIDR)notation, which puts a slash(/) after the dotted decimal notationfollowed by the number of 1-bits specified in the subnet mask. Subnetting provides more IP-address options for medium-sized networks, because there is a shortage of Class B and Class C addresses. In the Hands-on Projects 6-1 through 6-4 on pages 298 through 301you will learn how to determine IP address/Subnet Mask information for Windows, Linux, NetWare, and Mac OS X.
Border and Firewall Security
A Border (Border Gateway) is typically established between a private network (enterprise)
one used by a company for exampleand a public network, in particular the Internet.
For security, organizations establish border gateways at each border crossing. The border gatewayis a firewall that is configured with security policies to control the traffic that is permitted to cross a border in either direction.
The strongest border security design is to protect every border point, including:
Connection points between LANs and public or private WANS
Dial-up and cable modem access
Virtual private network (VPN) access
Short-range wireless access, including802.11Wireless and Bluetooth
Long-range wireless access, including satellite and microwave
Firewalls provide border security by using some or all of the following approaches:
Packet filtering
Network Address Translation (NAT)
application gateways or proxies
Packet Filtering
Packet filteringtypically involves using characteristics of TCP (or UDP) and IP to establish filters between two connected networks. Another type of packet filtering is to allow or block packets from specific protocols. For example, a packet filter might block NetBEUI protocol packets from an older Windows NT Server network, or it might block Internet Packet Exchange (IPX) protocol packets used by an older NetWare network. The IPX protocol was developed by Novell and was used extensively for versions of NetWare prior to version 5. A disadvantage of IPX is that it is a “chatty” protocol, because computers that use it frequently broadcast “I’m here” (hello) messages that can cause significant network traffic.
When you create a filter for TCP/IP, two important characteristics are the IP address information in a packet and the TCP or UDP port (socket) information. Another way to use a firewall is to control access across the firewall by TCP and/orUDPport number. Figure 6-7 on page 274 illustrates the use of a firewall to protect a specific subnet via the subnet identification and through port blocking.
Packet filtering is accomplished using one of two techniques: stateless filtering and stateful filtering. In stateless packet filtering the firewall examines every individual packet and decides whether to pass or block the packet, depending on the packet or segmentheader information. Stateful packet filtering, also called Stateful Packet Inspection (SPI), tracks information about a communication session, such as which ports/sockets are in use, drawing from the contents of multiple packets.
Network Address Translation (NAT)
When NAT is used, private IP-addresses on the network protected by NAT are seen by the outside world as a singlepublic IP-address, the WAN IP-address of the device configured for NAT, or as a public IP-addressselected from a pool of real but proxy IP-addresses. Using NAT discourages attackers, because they cannot identify a specific host to attack behind the NAT firewall device on the local internal network. Instead, the attacker sees only the global external IP-address used by the device running theNAT firewallsoftware (usually a router). Another advantage of NAT is that it enables a network to useIP-addresses on the internal local network that are not formally registered for Internet use (Private IP-Addresses). There are generally four ways to perform NAT translation:
Dynamic translation (or IP masquerade) Static translation
Network redundancy translation Load balancing
Proxies
A proxy is a host computer that is located between a host on an internal network and a host on an external network with which the internal host is communicating. A proxy can fulfill one or a combination of tasks:
Act as an application-level gateway
Filter communications
Create secure tunnels for communications
Enhance application request performance through caching
Proxies that are configured as application-level gateways can have different levels of ability. Some proxies function as circuit-level gateways, creating a virtual tunnel or Virtual Private Network (VPN) between the proxy and an external host. Some proxies are able to provide caching services as a way to reduce the load on servers within the internal network. Cache is storage used by a computer system to house frequently-used data in quickly accessed storage, such as memory.
Using Routers for Border Security
A routerperforms packet filtering and is often used as apolicyfirewall on a network, in addition to the other functions it performs. In general, routers are used to:
Efficiently direct packets from one network to another (routing)
Join neighboring or distant networks (VPNs)
Connect dissimilar networks (gateway)
Prevent network bottlenecks by isolating portions of a network (QoS)
Secure portions of a network from intruders (IDS)
Permit or deny packets on the basis of the source IP, the destination IP, the protocol(port/socket) used, and whether the packet is inbound or outbound(ACL)
A Routersrouting protocol (RIP, OSPF, BGP …) regularly exchanges informationabout network traffic, the network topology, and the status of network links. When a packet arrives, the router examines the protocol destination address, for instance the IP-address in a TCP/IP packet. The router thendetermines how to forward(route) the packet on the basis of the metrics it uses in the routing table (best path). A metric(cost value) is used to determine the best path/route through aninter-network (WAN).
Routers that are in a local systemfor example, within a single organization and on the same WAN (enterprise)use two common routing protocols for communications: RIP and OSPF. Routing Information Protocol (RIP) is used by routers to determine the fewest hops between themselves and other routers, and this information is added to each router’s routing table. Open Shortest Path First(OSPF) is more commonly used and offers several advantages over RIP. One advantage is that the router sends only the portion of the routing table that pertains to its most immediate change in router links.This is called the “link-state routing.” Two other advantages of OSPF are:
It packages routing information in a more compact packet format than RIP
Update routing link information is shared among routers, rather than the entire routing table.
Using the Firewall Capabilities in Operating Systems
Some operating systems allow you to configure firewall services for border security. A demilitarized zone(DMZ) is a separate portion of a network (a subnet) that exists between two or more networks that have different security measures in place, such as the “zone” between the private network of a company and the Internet. The advantage of placing servers in the DMZ is that the less-secure network communications required for access to the servers does not have to cross the border into the private network.
Configuring a Firewall in Windows XP Professional
When a host runningWindows XP Professionalis directly connected to the Internet, through a cable modem or DSL connection, the Internet Connection Firewall (ICF) should be enabled.SeeFigure 6-11 on page 284. You can also configure ICF for Internet Connection Sharing(ICS)(e.g. NAT) for hosts on a local area connection, particularly if these hosts are not already protected by a firewall. Once ICF is enabled, you can choose to allow or deny incoming services (protocol/port/socket), such as HTTP, HTTPS, FTP, SMTP, and others.
Quick Reference / Discuss what ICF is designed to do when it is enabled as shown on page 284.Configuring a Firewall in Windows Server
Windows Server 2003/2008 uses the same implementations of ICF as in Windows XP Professional. When you configure ICF for Windows Server, make sure that you enable only those services that are needed on the server; for instance, enable HTTP if you access web-pages on the Internet.
Not all versions of Windows Server 2003 come with ICF. It is packaged with Windows Server 2003, Standard Edition, and Windows Server 2003, Enterprise Edition. ICF is not available in:
64-bit versions of Windows Server 2003
Windows Server 2003, Datacenter Edition
Windows Server 2003, Web Edition
Configuring NAT in Windows Server 2003/2008
Windows Server 2003/2008 can be configured to provide NATfirewall services, for connections that go over the Internet. NAT is just one of several services that can be set up in Windows Server through Microsoft Routing and Remote Access Services(RRAS)(see Figure 6-13 on page 286). When you configure NAT in Windows Server 2003/2008, you can configure it to work with one or more NICs –to connectLAN(s)or WAN connections to the server, or both.
Configuring NAT in Windows 2000 Server
In Windows 2000 Server, you can enable NAT by setting up the Windows 2000 server as an Internet connection server in the Windows 2000 Server Routing and Remote Access tool. When you configure Windows 2000 Server to use NAT, it functions similarly to the NAT implementation for a small office described for Windows Server 2003. Hands-on Project 6-7 on pages 304 and 305shows how to configure NAT in Windows 2000 Server.
Configuring a Firewall in Linux
The simplest way to configure a firewall in Red Hat Linux is by using the Security Level Configuration tool. This tool offers three basic security levels:
High Medium No Firewall
Additionally, when you customize the firewall, you can allow or deny access to any combination of the following services:
WWW (HTTP) FTP
SSH DHCP
Mail (SMTP/POP3/IMAP) Telnet
Configuring NAT and a Firewall Using iptables in Linux
Linux also offers the powerfuliptablescapability for configuring NAT and complex firewall security from the command line in a terminal window. If you are configuring a server or you want to fine-tune your firewall security on a workstation, configure the firewall using iptables instead of using the more basic Security Level Configurationtool. Iptables enables configuration ofpacket filter rules. A set of rules is called a chain, and it is applied to packets containing specific information. Table 6-3 on page 289 shows a sample of the parameters that you can use with the iptables command.
Configuring a Mac OS X Firewall
Mac OS X comes with a firewall that you can configure to control access into and out of the operating system over a network or Internet connection. The Mac OS Xfirewall enables you to allow or deny network communications through TCP and UDP ports by first turning specific services on or off. The services that you can turn on or off are:
Personal file sharing Windows file sharing
Personal Web sharing Remote login – SSH
FTP access Remote Apple events
Printer sharing
You can turn the firewall on or off, so that it allows or denies incoming network communications to the configured services.
Discussion Questions
1)Discuss the importance and functionality of a subnet mask
2)Discuss the procedures used to configure an iptablesfirewall in Linux
Additional Activities
1)Have students chart the differences and similarities between the commonly used network protocols.
2)Have students chart the differences and similarities of configuring NAT for Linux and the Windows operating systems
Michael Palmer, Guide To Operating Systems SecurityPage 1 of 7
Thompson/Course Technology ©2004 ISBN: 0-619-16040-3