At the 2015 RSA Conference in San Francisco, we released the first edition of The DomainTools Report: A Profile of Malicious Domains.
In this report, we investigate the attributes of malicious domains connected to malware, spam, phishing, and botnets. Using an aggregation of blacklists and DomainTools’ data, we compared the bad actors’ preferences for TLDs, email domains, privacy providers, and hosting locations. We identified some key trends that should help to profile cybercriminal behavior.
Why did we create this report?
Much of the malicious activity on the Internet is classified and tracked in domain blacklists and reputation scores. But these do little to profile and predict cybercrime to proactively protect against domains that have yet to exhibit illicit behavior. Malicious actors often behave in a predictable manner, and the more thoroughly we profile that behavior, the better we can defend against them. With that purpose in mind, we analyzed domains from several popular blacklists. This report uses DomainTools’ leading Whois and DNS data to define attributes of those malicious domains and begin to create a profile of locations and privacy preferences of cybercriminals.
What did we learn?
Our comprehensive coverage of Whois records enables us to take a broad look at registration attributes of all domains. Overlaying the domain data with data on malicious activity gave us quantitative insights into where the malicious and innocuous domains “live,” logically as well as geographically.
For example, one of the attributes we analyzed was the email domain used to register domains. In particular, we compared free email domains such as gmail.com, yahoo.com, hotmail.com and their variants and international counterparts. The results were very interesting and both expected and unexpected. Gmail.com was used for the most domain registrations, malicious or not. But based on percentage, some Japanese free email providers were the most malicious, primarily due to a large quantity of spam originating from those domains.
For more information and analysis, download a copy of the full report.
We will also be planning a live webinar later this quarter to discuss the report, and to answer your questions about the data and results. In the meantime, if you have any questions, please email us at email@example.com.
In the first two installments of this series (Beyond Whois: The Domain Profile and Beyond Whois: IP Addresses Tell Many Tales), we looked at how there are a lot of datapoints in the DomainTools database that go beyond Whois and DNS records to help you find elusive answers to domain ownership and the connections among domains. In this entry, we’ll see how historical information can assist your investigations, too.
Lucy 1: The Missing Link
You’re probably familiar with AL 288-1. Or, more likely, you know this Australopithecus celebrity by the whimsical name “Lucy.” (I’m taking some metaphorical license here, because paleontologically speaking, Australopithecus is not technically considered the “missing link.”) This Lucy represents the way historical artifacts shed light on chains of connection; sometimes a “digital fossil” is the missing link that can advance an investigation.
Many attack attribution or website fraud investigations call for you to look back in time, and if you’ve explored our offerings, you’ll be aware that Whois History is just one of the ways you can do your time-traveling explorations. Hosting History provides insight into how the domain has evolved over time, based on a combination of canonical Whois datapoints (name server and registrar) and our IP address data. Screenshot History can do the same.
Earlier, we saw how Reverse IP can show connections between domains that otherwise don’t look connected–especially if the IP in question has a small number of domains on it. Even if the present hosting doesn’t show such connections, though, comparing the Hosting History entries for domains where you suspect a possible connection could help confirm or refute your hypothesis. A shared–and “small”–IP address could be your missing link.
One usage of Screenshot History that we particularly like for attribution work is to look at the threshold of when a domain went into privacy protection. If the “before” screenshots–those taken before domain privacy went into effect–closely match the “after” screenshots, then this raises confidence that the visible owner of the domain before privacy is the same individual or organization behind it after privacy was enabled.
Lucy 2: Using only 10% of our brains?
The 2014 sci-fi hit Lucy trades heavily on the old canard that “we only use 10% of our brains” (which is actually discredited by most neuroscientists, but nonetheless is a common assumption). By using only Whois/DNS data to conduct a digital forensics investigation, you’re missing out on a wealth of other information that can be crucial to finding important answers. If you’re not already using the information we’ve covered in this series, carve out some time to get familiar with the various “beyond Whois” tools. They might just solve some thorny puzzles.
DomainTools works tirelessly to build the world’s best database of Whois records, with coverage spanning all of the ccTLDs and each new gTLD as it comes online—not to mention the “big six” TLDs: com, net, org, biz, info, and us. We believe we have reason to claim that our coverage is unparalleled. But now we have taken it, as they say, to the next level: we have developed the world’s best Whois parsing engine.
Since parsed Whois has existed for a while, we knew we had to clear a high bar in order to claim market leadership. We believe we have and encourage you to try it and let us know what you think.
- Covers more domains than anyone else—270M+ including ccTLDs and new gTLDs
- Parses over 95% of Whois record formats—industry’s highest
- Is highly reliable thanks to state of the art DomainTools data centers
- Normalizes fields that vary in format such as dates and phone numbers
- Priced competitively
The Parsed Whois API is optimized to allow quick retrieval of a Whois record, with each data field parsed out separately for easy integration into your systems and applications. This is ideal for anyone wishing to search for, index, or cross-reference data from one or multiple Whois records without having to build their own text parser. It opens up nearly limitless possibilities for domain-based research, investigations, and analysis.
Why this is important
In short, because it enables faster, more efficient analysis of the data, because it is delivered in a structured and context-relevant format. People seek Whois data because they are interested in learning about individual domains, groups of domains, the people behind domains or connections between domains. Many kinds of domain-related investigations, such as threat intelligence, incident response or online fraud investigations, depend on this information.
If you are using Whois data for security investigations or incident response, you already know that you can’t afford stale, inaccurate or missing data. While we have long been the leader in Whois data with the best coverage and frequency, we can now make the (somewhat immodest, but indulge us!) claim of the world’s best Whois parsing engine, with the highest level of accuracy and data normalization. That’s how the best just got even better. Visit the Parsed Whois API page to learn more.
In Part 1 of our Beyond Whois blog, we introduced the concept of the Domain Profile, the set of datapoints that DomainTools returns in a Whois lookup. The Domain Profile gives a wealth of information that is not contained in the actual Whois record for a domain, so using DomainTools gives you an investigative starting point with many more potential leads than a basic Whois lookup. In this installment, we’ll look at how the “beyond Whois” datapoints can assist your investigations.
First, to set context, we’ll define some terms which come up frequently in cybercrime investigations (and, with sometimes different terminology, in other kinds of investigations):
- Attribution: in the simplest terms, attribution is the process of naming the initiator of an activity, especially in cases where the person or organization being investigated does not want to be identified. A common example is finding the owner of a malicious domain.
- Enumeration (also called Forensic Domain Mapping): discovering the extent of a given individual or organization’s holdings.
Many different investigations boil down to one or both of these, and sometimes one serves the other. For example, by enumerating the holdings of a digital John Doe, one of the associated domains may have details leading to attribution that weren’t present in the original investigation.
Imagine that you want to learn who owns a domain that uses Whois privacy to mask the owner’s identity. If the website itself doesn’t appear to help either (i.e. there’s no “About Us” page with the info you’re seeking), you can dig in to the Domain Profile to propel your research forward. Per Part I of this blog series, start with the Whois record and Whois History to see if you can find attribution information directly. If not—and if they are worth their salt, you won’t—then a good next step is with the IP address.
By itself, an IP address doesn’t tell you much, but that’s where Domain Profile helps you:
- You can see how many sites are hosted on this IP address via Reverse IP. Why does this matter? If the IP address has many domains (thousands, even tens of thousands), then this may not be your most productive path to follow. But what if it hosts only a handful of sites? This increases the odds that there’s some kind of connection among those sites, because a large hosting provider would be unlikely to allocate an address to only a random handful of domains. Take a look at the other sites on the IP address. Do they have some characteristic that may link them to the original target domain? If so, look at the Whois records from these co-hosted domains. Perhaps one of them contains the gold nugget you are seeking: the actual identity of your digital John Doe!
- You can also see where the IP address is located. If a site that purports to sell Seattle Seahawks official merchandise, for example, is hosted in Uzbekistan, yet represents itself to be “direct from Seattle,” this can help characterize the domain. If you have a few domains that you think might be connected, look for a related pattern like this one, where the locus of business and the locus of hosting don’t make any rational sense. (Do be aware, however, that IP location OFTEN doesn’t match physical location, and in many or even most cases, this is normal and innocuous. Your own intuition and judgment will be helpful in this regard. A Seahawks merch vendor with an IP address in Virginia is not nearly as suspicious from an investigative standpoint as one in Uzbekistan.)
- You can see who owns the IP address by doing an IP Whois lookup. This will tell you whether the address is owned by a reputable, big hosting provider or ISP, or a smaller one. So-called “bulletproof hosting” sites are of particular interest to cybercrime investigators, because these hosters aim to shield their customers from prosecution, law enforcement takedown activities, sinkholing, and other measures taken by crime fighters, and to circumvent legal restrictions on what materials can be uploaded/hosted. Bulletproof hosting has been home to many of the world’s largest spammers and phishers.
When you use Reverse IP to see what other sites are hosted on the same IP, you may want to use the screenshot from Domain Profile to look for similarities. Especially if the domain(s) you’re researching are known for malware or other dangers, it’s often prudent not to visit the sites themselves.
Many attack attribution or website fraud investigations call for you to look back in time, and in our next installment, we’ll see how the historical data that is summarized in the Domain Profile can advance your investigation.
Until next time, happy exploring!
Official ownership records are valuable and can often tell interesting tales about the goods–physical or digital–that they cover. However, ownership records only go so far, as anyone who has used a commercial vehicle history report knows! Getting beyond the basics of registration data can make a world of difference for prospective buyers, and several firms have made a lucrative business out of providing detailed histories of cars and light trucks.
Internet domains are no different. It’s a safe bet that almost anyone reading this is very well-versed with Whois (which used to be spelled WHOIS, in the stilted English of protocol names back in the day). That familiar Courier-font blob of domain registration information is key to all kinds of activities, from domain investment and management, to brand management, to cybercrime investigation. The vast majority of investigations at DomainTools begin with a Whois lookup.
As useful as this information is, however, there is much more that can be learned about a domain by going beyond the data in the Whois record itself, so we’re going to spend some time looking at what’s *not* in the Whois records.
Here at DomainTools, we’ve been collecting and presenting such additional data for years, and the extra information forms a large part of the structure and experience of our Whois lookup results. We call this combination of Whois registration data and additional domain information the Domain Profile (though that name is not explicitly shown on the Whois results page).
Here are examples of the Domain Profile information, the datapoints that go beyond the Whois record:
- IP address (some domains may not have one associated with them, but most do, even if it’s just a parking site from the registrar)
- IP geolocation location and ASN (this tells you about the network on which the domain resides)
- Website title
- Response code (the code the web server sends back upon the initial HTTP connection–assuming the domain has a website up and running)
- Server type
- SEO score, terms, GA codes, images, and links information
- MX records (these are not on the Whois results page, but are available to you through Reverse MX)
Each of these pieces of information can be very valuable. Which are most important depends on the type of investigation you are conducting. But right from the get-go, they help to give you an overall sense of the status of the domain, allowing you to very quickly assess the basics:
- Does it have a website? Does the site look “professional?” Does it look as though it’s been updated recently?
- Does it reside on a dedicated IP address, or a relatively “small” IP address (one with not too many other domains on it), or is it on a big hosting site?
- Is there evidence that the domain’s owner has tried to maximize the domain’s profile, through SEO and other optimization techniques?
- How does the owner (or at least the webmaster) describe the web site? What are they trying to tell the world (and search engine bots!) about the site?
By spending as little as a few seconds looking over the Domain Profile on the Whois results page, you can pick up a lot of useful detail, which in turn informs your decisions about what to do next in your investigation. The next blog on “Beyond Whois” will give more detail on how the Domain Profile datapoints can point you toward valuable answers and sometimes-unexpected insights.
As always, we invite your questions and feedback to firstname.lastname@example.org. Thanks for reading, and happy exploring!
If you’ve spent any time on our site, you already know that “Whois” with DomainTools is much more than just a static Whois entry for a domain. Our Whois results page provides a detailed profile of the domain, including summary information on related IP addresses, name servers, IP geolocation, and web server stats and historical stats on domain ownership, NS, IP and Screenshot changes.
For most of our users, this is the jumping-off point for all kinds of investigations into cybercrime, security threats, online fraud, domains purchase, domain value, marketing opportunities, competition or any number of other activities.
We’ve just made access to this information easier to use, cleaner and faster. We’ve done a major overhaul of the Whois results page—a page that hasn’t had a design/UI update in many years. Not only was our goal to make it cleaner, better, faster, but to make it easier for users to start their investigations and leverage our Reverse, History, Monitors and other premium products. Like with a beautiful old building, at some point you need to renovate the lobby to make it look clean and function better.
Here’s an overview of the changes:
- Major profile information has been cleaned up and organized so that the most valued and used information is on top.
- Rows that contain historical or reverse lookup information such as registrant email, Whois History, and IP address, have action buttons that let you instantly “pivot” on that data point.
- We flattened the tab structure so that key information is visible at a glance. We’ve included expand/collapse controls for the major sections of the table, as well.
- A new “Tools” section is available in the upper right enabling you to jump straight to specific tools, view the domain’s screenshot, buy/backorder the domain, etc.
- We’ve decreased load time.
For details on the changes and how to get the most out of the new features, read the User Guide.
What hasn’t changed:
- We still provide the best coverage of Whois reporting across ccTLDs, new TLDs and gTLDs.
- We still provide Whois history back 12 years.
- We still provide the industry’s best Reverse Whois, Reverse IP, Reverse Name Server and Reverse Mail Server products.
- We still provide the #1 Domain Search, domain name typo and name spinning products.
- We still provide 100% real-time lookups, every time, for paying members. For non-paying members, most lookups are real-time and never older than that day. (The first lookup of every domain per day are real-time and subsequent lookups are never older than that day.)
- The raw Whois record is always provided.
The new Whois results page is a big change from the previous. For long-time users, there might be a bit of a learning curve. So, please, jump in, explore, experiment and get to know it. We believe you will find it much more efficient. If, after you’ve tried it for a few days, you still have suggestions on how to make it better, please send us your feedback at email@example.com.
For casual visitors, many of our more powerful tools, like Reverse Whois, Whois History and Domain Search, are available only to subscribing members. If you’re not already a DomainTools Professional member, sign up for a membership or a Free Trial to find out for yourself how much you can learn with the research products we’ve assembled on top of the world’s largest database of domain and IP information.
Thanks for using DomainTools and happy exploring!
This week, DomainTools will redirect whois.sc traffic to whois.domaintools.com. Many of you will remember that DomainTools originally started as “Whois.sc”. And that URL has maintained its sole purpose of serving the best Whois records in the industry. But as we continue to build and refine our business maintaining multiple sites and brands has created complexity. We’re in the process of revamping the entire whois experience on DomainTools.com and want to bring all of our loyal users under one umbrella.
For several years now, the whois searches at Whois.sc have returned results from whois.domaintools.com/[domain.tld] pages. It is only the whois.sc homepage that is now being redirected as the final piece in the migration. By redirecting Whois.sc we can focus all our resources on a single site and are able to offer you, our customers, a more integrated, feature rich and modern user experience.
If you are among the few that still use Whois.sc as your first interaction with us, we hope this is not a great inconvenience and that you find using whois.domaintools.com just as easy, and hopefully better. If not, we are always interested in hearing what we can do to make your experience better. Thank you for your understanding.
When we rolled out our new website home page last November, we promised a transformation of our design and User Experience (UX) across our entire site. And we’re not settling for superficial design changes, we are rethinking our UX to align with our users’ experiences and what they are trying to accomplish with our products.
One of the most powerful products DomainTools offers its members is our unique Whois History. Many of you visit Whois History every day—some, many times per day! This product contains valuable insight that can power many kinds of investigations, including researching the ownership history of a domain you wish to own, tracking down cyber-criminals and fraudsters before they hid behind Whois privacy protection and gathering evidence of ownership and usage of a given website.
Today, we’ve made all of that much easier. We’ve redesigned the Whois History UX to be more intuitive and to provide a huge boost in functionality in order to accelerate your research. The new Whois History makes it faster for you to pinpoint significant events in the history of a domain, such as ownership, Whois privacy, and contact information changes. This will dramatically reduce the amount of time required to investigate historical changes to a domain.
What’s changed? Take a look!
- Total layout and UX redesign—This is not mere window dressing! The new look enables you to quickly find dates with changed records, while viewing and navigating the data at the same time.
- Filtering for fast search—Filtering helps you narrow a search and pull out those records that contain a specific bit of information, such as a person or organization name, phone number, physical address, etc.
- Whois record changes highlighted—You can now view the specific changes that occurred highlighted within the documents themselves. No more hunting back and forth between two records to find the differences!
- Screenshots with Whois records—Links to screenshots were added so you can track homepage changes at the time of the Whois record change without leaving the page.
- Inline Reverse Whois lookups
- Download and Bookmark records
- …and, importantly, we have not removed any functionality. You’ll still be able to accomplish everything you always could—and more!
The new site should be easy to figure out, but if you want to accelerate your ramp to becoming a Whois History Power User, all the new features are explained in this help page and video link:
We are very excited about this update. Everything we do at DomainTools is aimed at helping you get the answers you need quickly and efficiently. We believe this update to Whois History delivers on that promise.
VP of Product
We are very excited to present a new look and feel for DomainTools’ website! This is the first of many changes to design, usability and architecture to come. We have long wanted to update the look and usability of our website, but have prioritized delivering the best data, scalable and reliable infrastructure and innovative products over UI as we know that access to data is what drives value for you, our customers.
We are now undergoing a UI and usability refresh to bring our design up with today’s standards and ensure an efficient and positive user experience. This home page and navigation redesign is just Phase One and includes major improvements to design, navigation, overall site usability and support resources architecture to make our customer’s experience better.
As I’m sure is apparently obvious, our new home page and solution pages reflect a significant new growth opportunity in our business; Enterprise sales for customers in the brand protection, cybercrime investigation and ad or social network markets. As the leaders in domain name and DNS data intelligence, DomainTools holds incredible value to companies who want to use “internet data” to know who is attacking their networks or infringing on their brand on the Internet. This evolution benefits everyone, as new revenue growth enables us to invest more in product innovation and data gathering.
We remain committed to the customers on which this Company was founded: Domain professionals. While a home page and site redesign is a natural first step of a website overhaul, future improvements will bring much needed updates to our core Whois and domain research usability and provide valuable new research tools and data visualization elements. Keep a close eye on the site, and this blog, as we rollout improvements as they are ready.
- Design and navigation, across the entire site
- Re-architected support page for better search and easier access to help, product information and best practice resources
- New Solution Briefs and education collateral
- New “DomainTools Labs” section showcasing Nextgen product innovation
- Peek at upcoming products IP Monitor and Reverse IP Whois
We hope you like the new improvements. I know the new design and logo will come as a shock for those who have gotten used to our look and feel over the last 6 years, but change is good and reflects our commitment to investing in the business. Look for more as we continue to rollout updates that make it easier for you, our members, to do the research and get the information you need.