8.9.0

- Add source name and type information based on static mapping of the reverse DNS base domain
  - See [this documentation](https://github.com/domainaware/parsedmarc/tree/master/parsedmarc/resources/maps) for more information, and to learn how to help!
- Replace `multiprocessing.Pool` with `Pipe` + `Process` (PR #491 closes issue #489)
- Remove unused parallel arguments (PR #492 closes issue #490)

.gitignore

@@ -113,7 +113,6 @@ ENV/
 
 # I/O files
 output/
-*.csv
 *.xls*
 
 # LibreOffice lock files

CHANGELOG.md

@@ -1,6 +1,14 @@
 Changelog
 =========
 
+8.9.0
+-----
+
+- Add source name and type information based on static mapping of the reverse DNS base domain
+  - See [this documentation](https://github.com/domainaware/parsedmarc/tree/master/parsedmarc/resources/maps) for more information, and to learn how to help!
+- Replace `multiprocessing.Pool` with `Pipe` + `Process` (PR #491 closes issue #489)
+- Remove unused parallel arguments (PR #492 closes issue #490)
+
 8.8.0
 -----
 

parsedmarc/__init__.py

@@ -34,7 +34,7 @@
 from parsedmarc.utils import parse_email
 from parsedmarc.utils import timestamp_to_human, human_timestamp_to_datetime
 
-__version__ = "8.8.0"
+__version__ = "8.9.0"
 
 logger.debug("parsedmarc v{0}".format(__version__))
 

parsedmarc/resources/maps/README.md

@@ -1,47 +1,50 @@
 # About
 
-These files are meant to make it easier to identify who or what a sending source is. Please consider contributing
+A mapping is meant to make it easier to identify who or what a sending source is. Please consider contributing
 additional mappings in a GitHub Pull Request.
 
 ## base_reverse_dns_map.csv
 
-A CSV file with three fields: `base_reverse_dns`, `service_name`, and `service_type`.
+A CSV file with three fields: `base_reverse_dns`, `name`, and `type`.
 Most of the time the base reverse DNS of sending service is closely related to the name of the
 service, but not always. Sometimes services will use multiple reverse DNS domains for the same service. For example,
 Intuit Mailchimp uses the base domains `mcdlv.net`, `mcsv.net`,
-and `rsgsv.net`. Having all of these mapped makes it easier to answer questions like: "How many emails is
-Intuit Mailchimp sending as my domains?"
+and `rsgsv.net`. Having all of these mapped to the same service name and type makes it easier to answer questions like:
+"How many emails is Intuit Mailchimp sending as my domains?"
 
-The `service_type` is based on the primary service provided by that entity. For example, most ISPs provide email
-hosting to their customers , but the primary purpose of the service is to provide internet access. Likewise, nearly all
-email `Marketing` services are `SaaS` platforms, but it is more useful to identify them as marketing platforms. For
-individual entities that use their own reverse DNS domain names but do not provide a `SaaS` platform, setting the 
-`service_type` to the industry is most useful, with the notable exception on `Email Security` services. The
-current `service_type` values in use are:
+The `service_type` is based on the following rule precedence:
 
-Email Provider
-Email Security
-Entertainment
-Finance
-Food
-Government
-Government Media
-Healthcare
-Industrial
-ISP
-Logistics
-Marketing
-MSP
-Nonprofit
-Print
-Real Estate
-Retail
-SaaS
-Social Media
-Technology
-Travel
-University
-Web Host
+1. All email security services are identified as `Email Security`, no matter how or where they are hosted.
+2. All marketing services are identified as `Marketing`, no matter how or where they are hosted.
+3. All telecommunications providers that offer internet access are identified as `ISP`, even if they also offer other services, such as web hosting or email hosting.
+4. All web hosting providers are identified as `Web Hosting`, even if the service also offers email hosting.
+5. All email account providers are identified as `Email Provider`, no matter how or where they are hosted
+6. All legitimate platforms offering their Software as a Service SaaS) are identified as `SaaS`, regardless of industry. This helps simplify metrics.
+7. All other senders that use their own domain as a Reverse DMS base domain should be identified based on their industry
+
+- Email Provider
+- Email Security
+- Entertainment
+- Finance
+- Food
+- Government
+- Government Media
+- Healthcare
+- Industrial
+- ISP
+- Logistics
+- Marketing
+- MSP
+- Nonprofit
+- Print
+- Real Estate
+- Retail
+- SaaS
+- Social Media
+- Technology
+- Travel
+- University
+- Web Host
 
 The file currently contains over 400 mappings from a wide variety of email sending services, including large email
 providers, SaaS platforms, small web hosts, and healthcare companies. Ideally this mapping will continuously grow to