Contents
PRODUCT END OF LIFE
1 Introduction
1.1 About the data feeds
1.2 Download schedule
1.2.1 When are the data provided
1.2.2 Normal timings
1.2.3 Unpredictable and irregular delays
1.2.4 Schedule information
1.2.5 Data correction processes
2 Feeds, download directory structures and data formats
2.1 Major and new gTLDs
2.2 The
.us
TLD
2.3 Supported and unsupported TLDs
2.4 On the data feeds containing changes
2.5 Legacy vs. current URLs
2.5.1 Legacy URLs
2.5.2 New generation URLs
2.6 Data feeds, URLs, directory structures and data formats
2.6.1 Feed:
domain_names_new
2.6.2 Feed:
domain_names_dropped
2.6.3 Feed:
domain_names_whois
2.6.4 Feed:
domain_names_whois_archive
2.6.5 Feed:
domain_names_whois2
2.6.6 Feed:
domain_names_whois_filtered_reg_country
2.6.7 Feed:
domain_names_whois_filtered_reg_country_archive
2.6.8 Feed:
domain_names_dropped_whois
2.6.9 Feed:
domain_names_dropped_whois_archive
2.6.10 Feed:
domain_names_diff_whois_filtered_reg_country2
2.6.11 Feed:
domain_names_whois_filtered_reg_country_noproxy
2.6.12 Feed:
domain_names_whois_filtered_reg_country_noproxy_archive
2.6.13 Feed:
whois_record_delta_whois
2.6.14 Feed:
whois_record_delta_whois_archive
2.6.15 Feed:
whois_record_delta_whois_weekly
2.6.16 Feed:
whois_record_delta_domain_names_change
2.6.17 Feed:
whois_record_delta_domain_names_change_archive
2.6.18 Feed:
whois_record_delta_domain_names_change_weekly
2.6.19 Feed:
ngtlds_domain_names_new
2.6.20 Feed:
ngtlds_domain_names_dropped
2.6.21 Feed:
ngtlds_domain_names_whois
2.6.22 Feed:
ngtlds_domain_names_whois_archive
2.6.23 Feed:
ngtlds_domain_names_dropped_whois
2.6.24 Feed:
ngtlds_domain_names_dropped_whois_archive
2.6.25 Feed:
ngtlds_domain_names_whois_filtered_reg_country
2.6.26 Feed:
ngtlds_domain_names_whois_filtered_reg_country_archive
2.6.27 Feed:
ngtlds_domain_names_whois_filtered_reg_country_noproxy
2.6.28 Feed:
ngtlds_domain_names_whois_filtered_reg_country_noproxy_archive
2.7 Supported tlds
2.8 Auxiliary data on actual time of data file creation
2.9 Data file hashes for integrity checking
3 CSV file formats
3.1 The use of CSV files
3.1.1 Loading CSV files into MySQL and other database systems
3.2 File formats
3.3 Data field details
3.4 Maximum data field lengths
3.5 Standardized country fields
4 JSON file availability
5 Database dumps
5.1 Software requirements for importing mysql dump files
5.2 Importing mysql dump files
5.2.1 Loading everything (including schema and data) from a single mysqldump file
5.3 Database schema
5.4 Further reading
6 Client-side scripts for downloading data, loading into databases, etc.
7 Tips for web-downloading data
7.1 When, how, and what to download
7.2 Downloaders with a GUI
7.3 Command-line downloaders
8 Handling large csv files
8.1 Line terminators in CSV files
8.2 Opening a large CSV file on Windows 8 Pro, Windows 7, Vista & XP
8.3 How can I open large CSV file on Mac OS X?
8.4 Tips for dealing with CSV files from a shell (any OS)
9 Daily data collection methodology
9.1 Domain life cycle and feed timings
9.2 Time data accuracy
10 Data quality check
10.1 Quality check: csv files
10.2 Quality check: MySQL dumps
11 Access via SSL Certifiate Authenticaton
11.1 Setup instructions
11.1.1 Microsoft Windows
11.1.2 Mac OS X
11.1.3 Linux
11.2 Accessible URLs
12 FTP access of WHOIS data
12.1 FTP clients
12.2 FTP access
12.3 FTP directory structure of legacy and quarterly subscriptions
12.4 FTP firewall settings for legacy subscriptions