Network Working Group L. Andersson
Request for Comments: 4948 Acreo AB
Category: Informational E. Davies
Folly Consulting
L. Zhang
UCLA
August 2007
Report from the IAB workshop on Unwanted Traffic March 9-10, 2006
Status of This Memo
This memo provides information for the Internet community. It does
not specify an Internet standard of any kind. Distribution of this
memo is unlimited.
Copyright Notice
Copyright (C) The IETF Trust (2007).
Abstract
This document reports the outcome of a workshop held by the Internet
Architecture Board (IAB) on Unwanted Internet Traffic. The workshop
was held on March 9-10, 2006 at USC/ISI in Marina del Rey, CA, USA.
The primary goal of the workshop was to foster interchange between
the operator, standards, and research communities on the topic of
unwanted traffic, as manifested in, for example, Distributed Denial
of Service (DDoS) attacks, spam, and phishing, to gain understandings
on the ultimate sources of these unwanted traffic, and to assess
their impact and the effectiveness of existing solutions. It was
also a goal of the workshop to identify engineering and research
topics that could be undertaken by the IAB, the IETF, the IRTF, and
the network research and development community at large to develop
effective countermeasures against the unwanted traffic.
Andersson, et al. Informational [Page 1]
RFC 4948 Unwanted Traffic August 2007
Table of Contents
1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . 3
2. The Root of All Evils: An Underground Economy . . . . . . . . 4
2.1. The Underground Economy . . . . . . . . . . . . . . . . . 5
2.2. Our Enemy Using Our Networks, Our Tools . . . . . . . . . 6
2.3. Compromised Systems Being a Major Source of Problems . . . 7
2.4. Lack of Meaningful Deterrence . . . . . . . . . . . . . . 8
2.5. Consequences . . . . . . . . . . . . . . . . . . . . . . . 10
3. How Bad Is The Problem? . . . . . . . . . . . . . . . . . . . 10
3.1. Backbone Providers . . . . . . . . . . . . . . . . . . . . 10
3.1.1. DDoS Traffic . . . . . . . . . . . . . . . . . . . . . 10
3.1.2. Problem Mitigation . . . . . . . . . . . . . . . . . . 11
3.2. Access Providers . . . . . . . . . . . . . . . . . . . . . 12
3.3. Enterprise Networks: Perspective from a Large
Enterprise . . . . . . . . . . . . . . . . . . . . . . . . 13
3.4. Domain Name Services . . . . . . . . . . . . . . . . . . . 14
4. Current Vulnerabilities and Existing Solutions . . . . . . . . 15
4.1. Internet Vulnerabilities . . . . . . . . . . . . . . . . . 15
4.2. Existing Solutions . . . . . . . . . . . . . . . . . . . . 16
4.2.1. Existing Solutions for Backbone Providers . . . . . . 16
4.2.2. Existing Solutions for Enterprise Networks . . . . . . 17
4.3. Shortfalls in the Existing Network Protection . . . . . . 18
4.3.1. Inadequate Tools . . . . . . . . . . . . . . . . . . . 18
4.3.2. Inadequate Deployments . . . . . . . . . . . . . . . . 18
4.3.3. Inadequate Education . . . . . . . . . . . . . . . . . 19
4.3.4. Is Closing Down Open Internet Access Necessary? . . . 19
5. Active and Potential Solutions in the Pipeline . . . . . . . . 20
5.1. Central Policy Repository . . . . . . . . . . . . . . . . 20
5.2. Flow Based Tools . . . . . . . . . . . . . . . . . . . . . 21
5.3. Internet Motion Sensor (IMS) . . . . . . . . . . . . . . . 21
5.4. BCP 38 . . . . . . . . . . . . . . . . . . . . . . . . . . 22
5.5. Layer 5 to 7 Awareness . . . . . . . . . . . . . . . . . . 22
5.6. How To's . . . . . . . . . . . . . . . . . . . . . . . . . 22
5.7. SHRED . . . . . . . . . . . . . . . . . . . . . . . . . . 23
6. Research in Progress . . . . . . . . . . . . . . . . . . . . . 23
6.1. Ongoing Research . . . . . . . . . . . . . . . . . . . . . 23
6.1.1. Exploited Hosts . . . . . . . . . . . . . . . . . . . 23
6.1.2. Distributed Denial of Service (DDoS) Attacks . . . . . 25
6.1.3. Spyware . . . . . . . . . . . . . . . . . . . . . . . 26
6.1.4. Forensic Aids . . . . . . . . . . . . . . . . . . . . 26
6.1.5. Measurements . . . . . . . . . . . . . . . . . . . . . 27
6.1.6. Traffic Analysis . . . . . . . . . . . . . . . . . . . 27
6.1.7. Protocol and Software Security . . . . . . . . . . . . 27
6.2. Research on the Internet . . . . . . . . . . . . . . . . . 27
6.2.1. Research and Standards . . . . . . . . . . . . . . . . 28
6.2.2. Research and the Bad Guys . . . . . . . . . . . . . . 29
Andersson, et al. Informational [Page 2]
RFC 4948 Unwanted Traffic August 2007
7. Aladdin's Lamp . . . . . . . . . . . . . . . . . . . . . . . . 30
7.1. Security Improvements . . . . . . . . . . . . . . . . . . 30
7.2. Unwanted Traffic . . . . . . . . . . . . . . . . . . . . . 31
8. Workshop Summary . . . . . . . . . . . . . . . . . . . . . . . 31
8.1. Hard Questions . . . . . . . . . . . . . . . . . . . . . . 31
8.2. Medium or Long Term Steps . . . . . . . . . . . . . . . . 32
8.3. Immediately Actionable Steps . . . . . . . . . . . . . . . 33
9. Terminology . . . . . . . . . . . . . . . . . . . . . . . . . 33
10. Security Considerations . . . . . . . . . . . . . . . . . . . 38
11. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . . 38
12. Informative References . . . . . . . . . . . . . . . . . . . . 39
Appendix A. Participants in the Workshop . . . . . . . . . . . . 40
Appendix B. Workshop Agenda . . . . . . . . . . . . . . . . . . . 41
Appendix C. Slides . . . . . . . . . . . . . . . . . . . . . . . 41
1. Introduction
The Internet carries a lot of unwanted traffic today. To gain a
better understanding of the driving forces behind such unwanted
traffic and to assess existing countermeasures, the IAB organized an
"Unwanted Internet Traffic" workshop and invited experts on different
aspects of unwanted traffic from operator, vendor, and research
communities to the workshop. The intention was to share information
among people from different fields and organizations, fostering an
interchange of experiences, views, and ideas between the various
communities on this important topic. The major goal of this workshop
was to stimulate discussion at a deep technical level to assess
today's situation in regards to:
o the kinds of unwanted traffic that are seen on the Internet,
o how bad the picture looks,
o who and where are the major sources of the problem,
o which solutions work and which do not, and
o what needs to be done.
The workshop was very successful. Over one and half days of
intensive discussions, the major sources of the unwanted traffic were
identified, and a critical assessment of the existing mitigation
tools was conducted. However, due to the limitation of available
time, it was impossible to cover the topic of unwanted traffic in its
entirety. Thus, for some of the important issues, only the surface
was touched. Furthermore, because the primary focus of the workshop
was to collect and share information on the current state of affairs,
it is left as the next step to attempt to derive solutions to the
Andersson, et al. Informational [Page 3]
RFC 4948 Unwanted Traffic August 2007
issues identified. This will be done in part as activities within
the IAB, the IETF, and the IRTF.
During the workshop, a number of product and company names were
cited, which are reflected in the report to a certain extent.
However, a mention of any product in this report should not be taken
as an endorsement of that product; there may well be alternative,
equally relevant or efficacious products in the market place.
This report is a summary of the contributions by the workshop
participants, and thus it is not an IAB document. The views and
positions documented in the report do not necessarily reflect IAB
views and positions.
The workshop participant list is attached in Appendix A. The agenda
of the workshop can be found in Appendix B. Links to a subset of the
presentations are provided in Appendix C; the rest of the
presentations are of a sensitive nature, and it has been agreed that
they will not be made public. Definitions of the jargon used in
describing unwanted traffic can be found in Section 9.
2. The Root of All Evils: An Underground Economy
The first important message this workshop would like to bring to the
Internet community's attention is the existence of an underground
economy. This underground economy provides an enormous amount of
monetary fuel that drives the generation of unwanted traffic. This
economic incentive feeds on an Internet that is to a large extent
wide open. The open nature of the Internet fosters innovations but
offers virtually no defense against abuses. It connects to millions
of mostly unprotected hosts owned by millions of mostly naive users.
These users explore and benefit from the vast opportunities offered
by the new cyberspace, with little understanding of its vulnerability
to abuse and the potential danger of their computers being
compromised. Moreover, the Internet was designed without built-in
auditing trails. This was an appropriate choice at the time, but now
the lack of traceability makes it difficult to track down malicious
activities. Combined with a legal system that is yet to adapt to the
new challenge of regulating the cyberspace, this means the Internet,
as of today, has no effective deterrent to miscreants. The
unfettered design and freedom from regulation have contributed to the
extraordinary success of the Internet. At the same time, the
combination of these factors has also led to an increasing volume of
unwanted traffic. The rest of this section provides a more detailed
account of each of the above factors.
Andersson, et al. Informational [Page 4]
RFC 4948 Unwanted Traffic August 2007
2.1. The Underground Economy
As in any economic system, the underground economy is regulated by a
demand and supply chain. The underground economy, which began as a
barter system, has evolved into a giant shopping mall, commonly
running on IRC (Internet Relay Chat) servers. The IRC servers
provide various online stores selling information about stolen credit
cards and bank accounts, malware, bot code, botnets, root accesses to
compromised hosts and web servers, and much more. There are DDoS
attack stores, credit card stores, PayPal and bank account stores, as
well as Cisco and Juniper router stores that sell access to
compromised routers. Although not everything can be found on every
server, most common tools used to operate in the underground economy
can be found on almost all the servers.
How do miscreants turn attack tools and compromised machines into
real assets? In the simplest case, miscreants electronically
transfer money from stolen bank accounts directly to an account that
they control, often in another country. In a more sophisticated
example, miscreants use stolen credit cards or PayPal accounts for
online purchases. To hide their trails, they often find remailers
who receive the purchased goods and then repackage them to send to
the miscreants for a fee. The miscreants may also sell the goods
through online merchandising sites such as eBay. They request the
payments be made in cashier checks or money orders to be sent to the
people who provide money laundering services for the miscreants by
receiving the payments and sending them to banks in a different
country, again in exchange for a fee. In either case, the
destination bank accounts are used only for a short period and are
closed as soon as the money is withdrawn by the miscreants.
The miscreants obtain private and financial information from
compromised hosts and install bots (a.k.a. zombies) on them. They
can also obtain such information from phishing attacks. Spam
messages mislead naive users into accessing spoofed web sites run by
the miscreants where their financial information is extracted and
collected.
The miscreants in general are not skilled programmers. With money,
however, they can hire professional writers to produce well phrased
spam messages, and hire coders to develop new viruses, worms,
spyware, and botnet control packages, thereby resupplying the
underground market with new tools that produce more unwanted traffic
on the Internet: spam messages that spread phishing attacks, botnets
that are used to launch DDoS attacks, click fraud that "earns" income
by deceiving online commercial advertisers, and new viruses and worms
that compromise more hosts and steal additional financial information
as well as system passwords and personal identity information.
Andersson, et al. Informational [Page 5]
RFC 4948 Unwanted Traffic August 2007
The income gained from the above illegal activities allows miscreants
to hire spammers, coders, and IRC server providers. Spammers use
botnets. Direct marketing companies set up dirty affiliate programs.
Some less than scrupulous banks are also involved to earn transaction
fees from moving the dirty money around. In the underground market,
everything can be traded, and everything has a value. Thus is
spawned unwanted traffic of all kinds.
The underground economy has evolved very rapidly over the past few
years. In the early days of bots and botnets, their activities were
largely devoted to DDoS attacks and were relatively easy to detect.
As the underground economy has evolved, so have the botnets. They
have moved from easily detectable behavior to masquerading as normal
user network activity to achieve their goals, making detection very
difficult even by vigilant system administrators.
The drive for this rapid evolution comes to a large extent from the
change in the intention of miscreant activity. Early virus attacks
and botnets were largely anarchic activities. Many were done by
"script kiddies" to disrupt systems without a real purpose or to
demonstrate the prowess of the attacker, for example in compromising
systems that were touted as "secure". Mirroring the
commercialization of the Internet and its increasing use for
e-business, miscreant activity is now mostly focused on conventional
criminal lines. Systems are quietly subverted with the goal of
obtaining illicit financial gain in the future, rather than causing
visible disruptions as was often the aim of the early hackers.
2.2. Our Enemy Using Our Networks, Our Tools
Internet Relay Chat (IRC) servers are commonly used as the command
and control channel for the underground market. These servers are
paid for by miscreants and are professionally supported. They are
advertised widely to attract potential consumers, and thus are easy
to find. The miscreants first talk to each other on the servers to
find out who is offering what on the market, then exchange encrypted
private messages to settle the deals.
The miscreants are not afraid of network operators seeing their
actions. If their activities are interrupted, they simply move to
another venue. When ISPs take actions to protect their customers,
revenge attacks are uncommon as long as the miscreants' cash flow is
not disturbed. When a botnet is taken out, they move on to the next
one, as there is a plentiful supply. However, if an IRC server is
taken out that disturbs their cash flow, miscreants can be ruthless
and severe attacks may follow. They currently have no fear, as they
know the chances of their being caught are minimal.
Andersson, et al. Informational [Page 6]
RFC 4948 Unwanted Traffic August 2007
Our enemies make good use of the Internet's global connectivity as
well as all the tools the Internet has developed. IRC servers
provide a job market for the miscreants and shopping malls of attack
tools. Networking research has produced abundant results making it
easier to build large scale distributed systems, and these have been
adopted by miscreants to build large size, well-controlled botnets.
Powerful search engines also enable one to quickly find readily
available tools and resources. The sophistication of attacks has
increased with time, while the skills required to launch effective
attacks have become minimal. Attackers can be hiding anywhere in the
Internet while attacks get launched on a global scale.
2.3. Compromised Systems Being a Major Source of Problems
The current Internet provides a field ripe for exploitation. The
majority of end hosts run vulnerable platforms. People from all
walks of life eagerly jump into the newly discovered online world,
yet without the proper training needed to understand the full
implications. This is at least partially due to most users failing
to anticipate how such a great invention can be readily abused. As a
result, the Internet has ended up with a huge number of compromised
hosts, without their owners being aware that it has happened.
Unprotected hosts can be compromised in multiple ways. Viruses and
worms can get into the system through exploiting bugs in the existing
operating systems or applications, sometimes even in anti-virus
programs. A phishing site may also take the opportunity to install
malware on a victim's computer when a user is lured to the site.
More recently, viruses have also started being propagated through
popular peer-to-peer file sharing applications. With multiple
channels of propagation, malware has become wide-spread, and infected
machines include not only home PCs (although they do represent a
large percentage), but also corporate servers, and even government
firewalls.
News of new exploits of vulnerabilities of Microsoft Windows
platforms is all too frequent, which leads to a common perception
that the Microsoft Windows platform is a major source of
vulnerability. One of the reasons for the frequent vulnerability
exploits of the Windows system is its popularity in the market place;
thus, a miscreant's investment in each exploit can gain big returns
from infecting millions of machines. As a result, each incident is
also likely to make headlines in the news. In reality, all other
platforms such as Linux, Solaris, and MAC OS for example, are also
vulnerable to compromises. Routers are not exempt from security
break-ins either, and using a high-end router as a DoS launchpad can
be a lot more effective than using a bundle of home PCs.
Andersson, et al. Informational [Page 7]
RFC 4948 Unwanted Traffic August 2007
Quietly subverting large numbers of hosts and making them part of a
botnet, while leaving their normal functionality and connectivity
essentially unimpaired, is now a major aim of miscreants and it
appears that they are being all too successful. Bots and the
functions they perform are often hard to detect and most of the time
their existence are not known to system operators or owners (hence,
the alternative name for hosts infected with bots controlled by
miscreants - zombies); by the time they are detected, it might very
well be too late as they have carried out the intended
(mal-)function.
The existence of a large number of compromised hosts is a
particularly challenging problem to the Internet's security. Not
only does the stolen financial information lead to enormous economic
losses, but also there has been no quick fix to the problem. As
noted above, in many cases the owners of the compromised computers
are unaware of the problem. Even after being notified, some owners
still do not care about fixing the problem as long as their own
interest, such as playing online games, is not affected, even though
the public interest is endangered --- large botnets can use multi-
millions of such compromised hosts to launch DDoS attacks, with each
host sending an insignificant amount of traffic but the aggregate
exceeding the capacity of the best engineered systems.
2.4. Lack of Meaningful Deterrence
One of the Internet's big strengths is its ability to provide
seamless interconnection among an effectively unlimited number of
parties. However, the other side of the same coin is that there may
not be clear ways to assign responsibilities when something goes
wrong. Taking DDoS attacks as an example, an attack is normally
launched from a large number of compromised hosts, the attack traffic
travels across the Internet backbone to the access network(s) linking
to the victims. As one can see, there are a number of independent
stake-holders involved, and it is not immediately clear which party
should take responsibility for resolving the problem.
Furthermore, tracking down an attack is an extremely difficult task.
The Internet architecture enables any IP host to communicate with any
other hosts, and it provides no audit trails. As a result, not only
is there no limit to what a host may do, but also there is no trace
after the event of what a host may have done. At this time, there is
virtually no effective tool available for problem diagnosis or packet
trace back. Thus, tracking down an attack is labor intensive and
requires sophisticated skills. As will be mentioned in the next
section, there is also a lack of incentive to report security
attacks. Compounded with the high cost, these factors make forensic
tracing of an attack to its root a rare event.
Andersson, et al. Informational [Page 8]
RFC 4948 Unwanted Traffic August 2007
In human society, the legal systems provide protection against
criminals. However, in the cyberspace, the legal systems are lagging
behind in establishing regulations. The laws and regulations aim at
penalizing the conduct after the fact. If the likelihood of
detection is low, the deterrence would be minimal. Many national
jurisdictions have regulations about acts of computer fraud and
abuse, and they often carry significant criminal penalties. In the
US (and many other places), it is illegal to access government
computers without authorization, illegal to damage protected
government computers, and illegal to access confidential information
on protected computers. However, the definition of "access" can be
difficult to ascertain. For example, is sending an ICMP (Internet
Control Messaging Protocol) packet to a protected computer considered
illegal access? There is a lack of technical understanding among
lawmakers that would be needed to specify the laws precisely and
provide effective targeting limited to undesirable acts. Computer
fraud and liabilities laws provide a forum to address illegal access
activities and enable prosecution of cybercriminals. However, one
difficulty in prosecuting affiliate programs using bot infrastructure
is that they are either borderline legal, or there is little
evidence. There is also the mentality of taking legal action only
when the measurable monetary damage exceeds a high threshold, while
it is often difficult to quantify the monetary damage in individual
cases of cyberspace crimes.
There is a coalition between countries on collecting cybercriminal
evidence across the world, but there is no rigorous way to trace
across borders. Laws and rules are mostly local to a country,
policies (when they exist) are mostly enacted and enforced locally,
while the Internet itself, that carries the unwanted traffic,
respects no borders. One estimate suggests that most players in the
underground economy are outside the US, yet most IRC servers
supporting the underground market may be running in US network
providers, enjoying the reliable service and wide connectivity to the
rest of the world provided by the networks.
In addition, the definition of "unwanted" traffic also varies between
different countries. For example, China bans certain types of
network traffic that are considered legitimate elsewhere. Yet
another major difficulty is the trade-off and blurred line between
having audit trails to facilitate forensic analysis and to enforce
censorship. The greater ability we build into the network to control
traffic, the stronger would be the monitoring requirements coming
from the legislators.
It should be emphasized that, while a legal system is necessary to
create effective deterrence and sanctions against miscreants, it is
by no means sufficient on its own. Rather, it must be accompanied by
Andersson, et al. Informational [Page 9]
RFC 4948 Unwanted Traffic August 2007
technical solutions to unwanted traffic detection and damage
recovery. It is also by no means a substitute for user education.
Only a well informed user community can collectively establish an
effective defense in the cyberspace.
2.5. Consequences
What we have today is not a rosy picture: there are
o big economic incentives and a rich environment to exploit,
o no specific party to carry responsibility,
o no auditing system to trace back to the sources of attacks, and
o no well established legal regulations to punish offenders.
The combination of these factors inevitably leads to ever increasing
types and volume of unwanted traffic. However, our real threats are
not the bots or DDoS attacks, but the criminals behind them.
Unwanted traffic is no longer only aiming for maximal disruption; in
many cases, it is now a means to illicit ends with the specific
purpose of generating financial gains for the miscreants. Their
crimes cause huge economic losses, counted in multiple billions of
dollars and continuing.
3. How Bad Is The Problem?
There are quite a number of different kinds of unwanted traffic on
the Internet today; the discussions at this workshop were mainly
around DDoS traffic and spam. The impact of DDoS and spam on
different parts of the network differs. Below, we summarize the
impact on backbone providers, access providers, and enterprise
customers, respectively.
3.1. Backbone Providers
Since backbone providers' main line of business is packet forwarding,
the impact of unwanted traffic is mainly measured by whether DDoS
traffic affects network availability. Spam or malware is not a major
concern because backbone networks do not directly support end users.
Router compromises may exist, but they are rare events at this time.
3.1.1. DDoS Traffic
Observation shows that, in the majority of DDoS attacks, attack
traffic can originate from almost anywhere in the Internet. In
particular, those regions with high speed user connectivity but
Andersson, et al. Informational [Page 10]
RFC 4948 Unwanted Traffic August 2007
poorly managed end hosts are often the originating sites of DDoS
attacks. The miscreants tend to find targets that offer maximal
returns with minimal efforts.
Backbone networks in general are well-provisioned in regard to
traffic capacities. Therefore, core routers and backbone link
capacity do not get affected much by most DDoS attacks; a 5Gbps
attack could be easily absorbed without causing noticeable impact on
the performance of backbone networks. However, DDoS attacks often
saturate access networks and make a significant impact on customers.
In particular, multihomed customers who have multiple well-
provisioned connections for high throughput and performance may
suffer from aggregated DDoS traffic coming in from all directions.
3.1.2. Problem Mitigation
Currently, backbone networks do not have effective diagnosis or
mitigation tools against DDoS attacks. The foremost problem is a
lack of incentives to deploy security solutions. Because IP transit
services are a commodity, controlling cost is essential to surviving
the competition. Thus, any expenditure tends to require a clearly
identified return-on-investment (ROI). Even when new security
solutions become available, providers do not necessarily upgrade
their infrastructure to deploy the solutions, as security solutions
are often prevention mechanisms that may not have an easily
quantifiable ROI. To survive in the competitive environment in which
they find themselves, backbone providers also try to recruit more
customers. Thus, a provider's reputation is important. Due to the
large number of attacks and inadequate security solution deployment,
effective attacks and security glitches can be expected. However, it
is not in a provider's best interest to report all the observed
attacks. Instead, the provider's first concern is to minimize the
number of publicized security incidents. For example, a "trouble
ticket" acknowledging the problem is issued only after a customer
complains. An informal estimate suggested that only about 10% of
DDoS attacks are actually reported (some other estimates have put the
figure as low as 2%). In short, there is a lack of incentives to
either report problems or deploy solutions.
Partly as a consequence of the lack of incentive and lack of funding,
there exist few DDoS mitigation tools for backbone providers.
Network operators often work on their own time to fight the battle
against malicious attacks. Their primary mitigation tools today are
Access Control Lists (ACL) and BGP (Border Gateway Protocol) null
routes to black-hole unwanted traffic. These tools can be turned on
locally and do not require coordination across administrative
domains. When done at, or near, DDoS victims, these simple tools can
have an immediate effect in reducing the DDoS traffic volume.
Andersson, et al. Informational [Page 11]
RFC 4948 Unwanted Traffic August 2007
However, these tools are rather rudimentary and inadequate, as we
will elaborate in Section 4.2.1.
3.2. Access Providers
A common issue that access providers share with backbone providers is
the lack of incentive and the shortage of funding needed to deploy
security solutions. As with the situation with security incidents on
the backbone, the number of security incidents reported by access
providers is estimated to be significantly lower than the number of
the actual incidents that occurred.
Because access providers are directly connected to end customers,
they also face unique problems of their own. From the access
providers' viewpoint, the most severe impact of unwanted traffic is
not the bandwidth exhaustion, but the customer support load it
engenders. The primary impact of unwanted traffic is on end users,
and access providers must respond to incident reports from their
customers. Today, access providers are playing the role of IT help
desk for many of their customers, especially residential users.
According to some access providers, during the Microsoft Blaster worm
attack, the average time taken to handle a customer call was over an
hour. Due to the high cost of staffing the help desks, it is
believed that, if a customer calls the help desk just once, the
provider would lose the profit they would otherwise have otherwise
made over the lifetime of that customer account.
To reduce the high customer service cost caused by security breaches,
most access providers offer free security software to their
customers. It is much cheaper to give the customer "free" security
software in the hope of preventing system compromises than handling
the system break-ins after the event. However, perhaps due to their
lack of understanding of the possible security problems they may
face, many customers fail to install security software despite the
free offer from their access providers, or even when they do, they
may lack the skill needed to configure a complex system correctly.
What factors may influence how quickly customers get the security
breaches fixed? Past experience suggests the following observations:
o Notification has little impact on end user repair behavior.
o There is no significant difference in terms of repair behavior
between different industries or between business and home users.
o Users' patching behavior follows an exponential decay pattern with
a time constant of approximately 40% per month. Thus, about 40%
of computers tend to be patched very quickly when a patch is
Andersson, et al. Informational [Page 12]
RFC 4948 Unwanted Traffic August 2007
released, and approximately 40% of the remaining vulnerable
computers in each following month will show signs of being
patched. This leaves a few percent still unpatched after 6
months. In the very large population of Internet hosts, this
results in a significant number of hosts that will be vulnerable
for the rest of their life.
o There is a general lack of user understanding: after being
compromised, unmanaged computers may get replaced rather than
repaired, and this often results in infections occurring during
the installation process on the replacement.
3.3. Enterprise Networks: Perspective from a Large Enterprise
The operators of one big enterprise network reported their experience
regarding unwanted traffic to the workshop. Enterprises perceive
many forms of bad traffic including worms, malware, spam, spyware,
Instant Messaging (IM), peer-to-peer (P2P) traffic, and DoS.
Compared to backbone and access providers, enterprise network
operators are more willing to investigate security breaches, although
they may hesitate to pay a high price for security solutions. False
positives are very costly. Most operators prefer false negatives to
false positives. In general, enterprises prefer prevention solutions
to detection solutions.
Deliberately created unwanted traffic (as opposed to unwanted traffic
that might arise from misconfiguration) in enterprise networks can be
sorted into three categories. The first is "Nuisance", which
includes unwanted traffic such as spam and peer-to-peer file sharing.
Although there were different opinions among the workshop
participants as to whether P2P traffic should, or should not, be
considered as unwanted traffic, enterprise network operators are
concerned not only that P2P traffic represents a significant share of
the total network load, but they are also sensitive to potential
copyright infringement issues that might lead to significant
financial and legal impacts on the company as a whole. In addition,
P2P file sharing applications have also became a popular channel for
malware propagation.
The second category of unwanted traffic is labeled "Malicious", which
includes the traffic that spreads malware. This class of traffic can
be small in volume but the cost from the resulting damage can be
high. The clean up after an incident also requires highly skilled
operators.
The third category of unwanted traffic is "Unknown": it is known that
there exists a class of traffic in the network that can be best
described in this way, as no one knows its purpose or the locations
Andersson, et al. Informational [Page 13]
RFC 4948 Unwanted Traffic August 2007
of the sources. Malicious traffic can be obscured by encryption,
encapsulation, or covered up as legitimate traffic. The existing
detection tools are ineffective for this type of traffic. Noisy
worms are easy to identify, but stealth worms can open a backdoor on
hosts and stay dormant for a long time without causing any noticeable
detrimental effect. This type of bad traffic has the potential to
make the greatest impact on an enterprise from a threat perspective.
There are more mitigation tools available for enterprise networks
than for backbone and access network providers; one explanation might
be the greater affordability of solutions for enterprise networks.
The costs of damage from a security breach can also have a very
significant impact on the profits of an enterprise. At the same
time, however, the workshop participants also expressed concerns
regarding the ongoing arms race between security exploits and
patching solutions. Up to now, security efforts have, by and large,
been reactive, creating a chain of security exploits and a consequent
stream of "fixes". Such a reactive mode has not only created a big
security market, but also does not enable us to get ahead of
attackers.
3.4. Domain Name Services
Different from backbone and access providers, there also exists a
class of Internet service infrastructure providers. Provision of
Domain Name System (DNS) services offers an example here. As
reported by operators from a major DNS hosting company, over time
there have been increasingly significant DDoS attacks on .com, .net
and root servers.
DNS service operators have witnessed large scale DDoS attacks. The
most recent ones include reflection attacks resulting from queries
using spoofed source addresses. The major damage caused by these
attacks are bandwidth and resource exhaustion, which led to
disruption of critical services. The peak rate of daily DNS
transactions has been growing at a much faster rate than the number
of Internet users, and this trend is expected to continue. The heavy
load on the DNS servers has led to increasing complexity in providing
the services.
In addition to intentional DDoS Attacks, some other causes of the
heavy DNS load included (1) well known bugs in a small number of DNS
servers that still run an old version of the BIND software, causing
significant load increase at top level servers; and (2)
inappropriately configured firewalls that allow DNS queries to come
out but block returning DNS replies, resulting in big adverse impacts
on the overall system. Most of such issues have been addressed in
the DNS operational guidelines drafted by the IETF DNS Operations
Andersson, et al. Informational [Page 14]
RFC 4948 Unwanted Traffic August 2007
Working Group; however, many DNS operators have not taken appropriate
actions.
At this time, the only effective and viable mitigation approach is
over-engineering the DNS service infrastructure by increasing link
bandwidth, the number of servers, and the server processing power, as
well as deploying network anycast. There is a concern about whether
the safety margin gained from over-engineering is, or is not,
adequate in sustaining DNS services over future attacks. Looking
forward, there are also a few new issues looming. Two imminent ones
are the expected widespread deployment of IPv6 whose new DNS software
would inevitably contain new bugs, and the DNS Security Extensions
(DNSSEC), which could potentially be abused to generate DDoS attacks.
4. Current Vulnerabilities and Existing Solutions
This section summarizes three aspects of the workshop discussions.
We first collected the major vulnerabilities mentioned in the
workshop, then made a summary of the existing solutions, and followed
up with an examination of the effectiveness, or lack of it, of the
existing solutions.
4.1. Internet Vulnerabilities
Below is a list of known Internet vulnerabilities and issues around
unwanted traffic.
o Packet source address spoofing: there has been speculation that
attacks using spoofed source addresses are decreasing, due to the
proliferation of botnets, which can be used to launch various
attacks without using spoofed source addresses. It is certainly
true that not all the attacks use spoofed addresses; however, many
attacks, especially reflection attacks, do use spoofed source
addresses.
o BGP route hijacking: in a survey conducted by Arbor Networks,
route hijacking together with source address spoofing are listed
as the two most critical vulnerabilities on the Internet. It has
been observed that miscreants hijack bogon prefixes for spam
message injections. Such hijacks do not affect normal packet
delivery and thus have a low chance of being noticed.
o Everything over HTTP: port scan attacks occur frequently in
today's Internet, looking for open TCP or UDP ports through which
to gain access to computers. The reaction from computer system
management has been to close down all the unused ports, especially
in firewalls. One result of this reaction is that application
designers have moved to transporting all data communications over
Andersson, et al. Informational [Page 15]
RFC 4948 Unwanted Traffic August 2007
HTTP to avoid firewall traversal issues. Transporting "everything
over HTTP" does not block attacks but has simply moved the
vulnerability from one place to another.
o Everyone comes from Everywhere: in the earlier life of the
Internet it had been possible to get some indication of the
authenticity of traffic from a specific sender based for example
on the Time To Live (TTL). The TTL would stay almost constant
when traffic from a certain sender to a specific host entered an
operators network, since the sender will "always" set the TTL to
the same value. If a change in the TTL value occurred without an
accompanying change in the routing, one could draw the conclusion
that this was potential unwanted traffic. However, since hosts
have become mobile, they may be roaming within an operator's
network and the resulting path changes may put more (or less) hops
between the source and the destination. Thus, it is no longer
possible to interpret a change in the TTL value, even if it occurs
without any corresponding change in routing, as an indication that
the traffic has been subverted.
o Complex Network Authentication: Network authentication as it is
used today is far too complex to be feasible for users to use
effectively. It will also be difficult to make it work with new
wireless access technologies.
A possible scenario envisages a customers handset that is
initially on a corporate wireless network. If that customer
steps out of the corporate building, the handset may get
connected to the corporate network through a GPRS network. The
handset may then roam to a wireless LAN network when the user
enters a public area with a hotspot. Consequently, we need
authentication tools for cases when the underlying data link
layer technology changes quickly, possibly during a single
application session.
o Unused Security Tools: Vendors and standards have produced quite a
number of useful security tools; however, not all, or even most,
of them get used extensively.
4.2. Existing Solutions
4.2.1. Existing Solutions for Backbone Providers
Several engineering solutions exist that operators can deploy to
defend the network against unwanted traffic. Adequate provisioning
is one commonly used approach that can diminish the impact of DDoS on
the Internet backbone. The solution that received most mentions at
the workshop was BCP 38 on ingress filtering: universal deployment of
Andersson, et al. Informational [Page 16]
RFC 4948 Unwanted Traffic August 2007
BCP 38 can effectively block DDoS attacks using spoofed source IP
addresses. At present, Access Control List (ACL) and BGP null
routing are the two tools most commonly used by network operators to
mitigate DDoS attacks. They are effective in blocking DDoS attacks,
especially when being applied at or near a victim's site.
Unfortunately, BCP 38 is not widely deployed today. BCP 38 may
require device upgrades, and is considered tedious to configure and
maintain. Although widespread deployment of BCP 38 could benefit the
Internet as a whole, deployment by individual sites imposes a certain
amount of cost to the site, and does not provide a direct and
tangible benefit in return. In other words, BCP 38 suffers from a
lack of deployment incentives.
Both BGP null routing and ACL have the drawback of relying on manual
configuration and thus are labor intensive. In addition, they also
suffer from blocking both attack and legitimate packets. There is
also a potential that some tools could back-fire, e.g., an overly
long ACL list might significantly slow down packet forwarding in a
router.
Unicast Reverse Path Filtering (uRPF), which is available on some
routers, provides a means of implementing a restricted form of BCP 38
ingress filtering without the effort of maintaining ACLs. uRPF uses
the routing table to check that a valid path back to the source
exists. However, its effectiveness depends on the specificity of the
routes against which source addresses are compared. The prevalence
of asymmetric routing means that the strict uRPF test (where the
route to the source must leave from the same interface on which the
packet being tested arrived) may have to be replaced by the loose
uRPF test (where the route may leave from any interface). The loose
uRPF test is not a guarantee against all cases of address spoofing,
and it may still be necessary to maintain an ACL to deal with
exceptions.
4.2.2. Existing Solutions for Enterprise Networks
A wide variety of commercial products is available for enterprise
network protection. Three popular types of protection mechanisms are
o Firewalls: firewalls are perhaps the most widely deployed
protection products. However, the effectiveness of firewalls in
protecting enterprise confidential information can be weakened by
spyware installed internally, and they are ineffective against
attacks carried out from inside the perimeter established by the
firewalls. Too often, spyware installation is a byproduct of
installing other applications permitted by end users.
Andersson, et al. Informational [Page 17]
RFC 4948 Unwanted Traffic August 2007
o Application level gateways: these are becoming more widely used.
However, because they require application-specific support, and in
many cases they cache all the in-flight documents, configuration
can be difficult and the costs high. Thus, enterprise network
operators prefer network level protections over layer-7 solutions.
o Anti-spam software: Anti-spam measures consume significant human
resources. Current spam mitigation tools include blacklists and
content filters. The more recent "learning" filters may help
significantly reduce the human effort needed and decrease the
number of both false positives and negatives.
A more recent development is computer admission control, where a
computer is granted network access if and only if it belongs to a
valid user and appears to have the most recent set of security
patches installed. It is however a more expensive solution. A major
remaining issue facing enterprise network operators is how to solve
the user vulnerability problem and reduce reliance on user's
understanding of the need for security maintenance.
4.3. Shortfalls in the Existing Network Protection
4.3.1. Inadequate Tools
Generally speaking, network and service operators do not have
adequate tools for network problem diagnosis. The current approaches
largely rely on the experience and skills of the operators, and on
time-consuming manual operations. The same is true for mitigation
tools against attacks.
4.3.2. Inadequate Deployments
The limited number of existing Internet protection measures have not
been widely deployed. Deployment of security solutions requires
resources which may not be available. It also requires education
among the operational community to recognize the critical importance
of patch installation and software upgrades; for example, a bug in
the BIND packet was discovered and fixed in 2003, yet a number of DNS
servers still run the old software today. Perhaps most importantly,
a security solution must be designed with the right incentives to
promote their deployment. Effective protection also requires
coordination between competing network providers. For the time
being, it is often difficult to even find the contact information for
operators of other networks.
A number of workshop participants shared the view that, if all the
known engineering approaches and bug fixes were universally deployed,
the Internet could have been enjoying a substantially reduced number
Andersson, et al. Informational [Page 18]
RFC 4948 Unwanted Traffic August 2007
of security problems today. In particular, the need for, and lack
of, BCP 38 deployment was mentioned numerous times during the
workshop. There is also a lack of enthusiasm about the routing
security requirements document being developed by the IETF RPSEC
(Routing Protocol Security) Working Group, which focuses heavily on
cryptographically-based protection requirements. Not only would
cryptographically-based solutions face the obstacle of funding for
deployment, but also they are likely to bring with them their own set
of problems.
4.3.3. Inadequate Education
There exists an educational challenge to disseminate the knowledge
needed for secure Internet usage and operations. Easily guessed
passwords and plaintext password transmission are still common in
many parts of the Internet. One common rumor claims that Cisco
routers were shipped with a default password "cisco" and this was
used by attackers to break into routers. In reality, operators often
configure Cisco routers with that password, perhaps because of the
difficulty of disseminating passwords to multiple maintainers. A
similar problem exists for Juniper routers and other vendors'
products.
How to provide effective education to the Internet user community at
large remains a great challenge. As mentioned earlier in this
report, the existence of a large number of compromised hosts is one
major source of the unwanted traffic problem, and the ultimate
solution to this problem is a well-informed, vigilant user community.
4.3.4. Is Closing Down Open Internet Access Necessary?
One position made at the workshop is that, facing the problems of
millions of vulnerable computers and lack of effective deterrence,
protecting the Internet might require a fundamental change to the
current Internet architecture, by replacing unconstrained open access
to the Internet with strictly controlled access. Although the
participants held different positions on this issue, a rough
consensus was reached that, considering the overall picture,
enforcing controlled access does not seem the best solution to
Internet protection. Instead, the workshop identified a number of
needs that should be satisfied to move towards a well protected
Internet:
o the need for risk assessment for service providers; at this time,
we lack a commonly agreed bar for security assurance;
o the need to add traceability to allow tracking of abnormal
behavior in the network, and
Andersson, et al. Informational [Page 19]
RFC 4948 Unwanted Traffic August 2007
o the need for liability if someone fails to follow recommended
practices.
Adding traceability has been difficult due to the distributed nature
of the Internet. Collaboration among operators is a necessity in
fighting cybercrimes. We must also pay attention to preparation for
the next cycle of miscreant activity, and not devote all our efforts
to fixing the existing problems. As discussed above, the current
reactive approach to security problems is not a winning strategy.
5. Active and Potential Solutions in the Pipeline
This section addresses the issues that vendors recognized as
important and for which there will be solutions available in the near
future.
There are a number of potential solutions that vendors are working
on, but are not yet offering as part of their product portfolio, that
will allegedly remedy or diagnose the problems described in
Section 4.1.
Inevitably, when vendors have or are about to make a decision on
implementing new features in their products but have not made any
announcement, the vendors are not willing to talk about the new
features openly, which limits what can be said in this section.
5.1. Central Policy Repository
One idea is to build a Central Policy Repository that holds policies
that are known to work properly, e.g., policies controlling from whom
one would accept traffic when under attack. This repository could,
for example, keep information on which neighbor router or AS is doing
proper ingress address filtering. The repository could also hold the
configurations that operators use to upgrade configurations on their
routers.
If such a repository is to be a shared resource used by multiple
operators, it will necessarily require validation and authentication
of the stored policies to ensure that the repository does not become
the cause of vulnerabilities. Inevitably, this would mean that the
information comes with a cost and it will only be viable if the sum
of the reductions in individual operators' costs is greater than the
costs of maintaining the repository.
Andersson, et al. Informational [Page 20]
RFC 4948 Unwanted Traffic August 2007
5.2. Flow Based Tools
A set of tools based on flow data is widely used to extract
information from both network and data link layers. Tools have been
built that can be used to find out the sources of almost any type of
traffic, including certain unwanted traffic. These flow-based tools
make it possible to do things like DDoS traceback, traffic/peering
analyses, and detection of botnets, worms, and spyware.
These tools monitor flows on the network and build baselines for what
is the "normal" behavior. Once the baseline is available, it is
possible to detect anomalous activity. It is easy to detect
variations over time, and decide if the variation is legitimate or
not. It is possible to take this approach further, typically
involving the identification of signatures of particular types of
traffic.
These flow-based tools are analogous to the "sonar" that is used by
navies to listen for submarines. Once a particular submarine is
identified, it is possible to record its sonar signature to be used
to provide rapid identification in the future when the same submarine
is encountered again.
Examples of existing tools include
Cisco IOS NetFlow