CHANGES
=======

* Tweak exception handling in open\_mail\_archive()
* Remove deprecated isort flag
* Run formatter after black version upgrade
* Update requirements
* Update media types
* Update libpff-python-ratom to new release
* Remove temporary test bits
* Skip test that times out on OSX in Travis
* Increase pytest verbosity
* Add test pypi server to tox config
* Fix imports
* Tweak the requirements for CI testing
* Tweak the requirements for CI testing
* Tweak the requirements for CI testing
* Use updated libpff-python-ratom from test.pypi.org
* Add XCode 12 to CI
* Edit CONTRIBUTING.md
* Edit CONTRIBUTING.md
* Fix typo
* Tweak CONTRIBUTING.md
* Add contributing draft
* Clarifications and link updates in README
* Update media types
* Use explicit encoding for eml export
* Minor README language fix
* Added message body and header export info

0.4.3
-----

* Update list of known spaCy models
* Fix imports
* Extend KeyboardInterrupt handling unit test
* Test message contents extraction in ratom report command
* Add optional message contents extraction to ratom report command
* DRY database code
* Validate message contents in unit test
* Small fixture optimization
* Add ability to peg unit tests to specific spaCy model versions
* Update dependencies
* Fix ratom messages unit test
* Add stub for standalone message extraction command
* Small unit test tweak
* Add unit test for PffArchive.get\_message\_body
* Sort results in sample mbox file fixture
* Extend mbox archive unit tests
* Update unit test
* Rename internal parameters
* Minor refactoring
* Pass along extra parameters for message body extraction
* Add CLI support for extracting message bodies and headers
* Message body extraction WIP
* Add unit test for cleanup\_message\_body()
* Skip 2 obsolete unit tests
* Add .github/stale.yml
* Add mime type guessing fallback
* Update media types
* Better filtering of notes data
* Strip notes/calendar data in oversize message bodies
* Inline attachments stripping WIP
* Update Archive base class interface
* Update media types
* Update MboxArchive class interface
* Move constants to their own module
* Inline attachments stripping WIP
* Update dependencies with beautifulsoup and striprtf
* Check message content size before serializing/processing

0.4.2
-----

* Log message tweaks
* Update media types
* Logging tweaks
* Adjust verbosity
* Update dependencies
* Extend unit test
* Minor tweaks
* Update media types
* Update unit test
* Make MboxArchive's instance attribute mailbox private
* Mime type detection refactoring
* truncate mime type to 80 chars in mime type error messages
* Log filepath in mime type error messages (#124)
* Switch fixtures for CLI tests
* Switch fixtures for entity extraction test
* Store message date in DB report

0.4.1
-----

* Minor test tweak
* Remove redundant progress bar update call

0.4.0
-----

* Update README.md
* README.md tweak
* Use actual digest values in eml export example
* Add eml export section to README.md
* Group eml exports per source file
* Update dependencies
* Simplify access to non-code data
* Add comment
* Add unit test for get\_attachment\_metadata()
* Comment tweak
* Mime type validation tweaks
* Add mime type validation to PffArchive.get\_attachment\_metadata() (#124)
* Mime type validation WIP
* Test extract\_message\_from\_archive() with mbox archive
* Update media types
* Add CLI unit test for ratom emldump
* Parametrize conditional raising in unit tests
* Add unit tests for CLI input validators
* Add exception handling to emldump subcommand
* Cast msg\_id to integer
* Fix default arg value in extract\_message\_from\_archive()
* Update media types and fix typo
* Peg pytest to 5.3.5
* eml export WIP
* Move CLI utility code to the core library
* Formatting
* Add jsonschema and update dependencies
* eml export WIP - Add json input validation
* eml export WIP
* eml export WIP
* Add get\_message\_by\_id() to base archive class

0.3.1
-----

* Update dependencies
* Add packaging to production dependencies
* Fixed a typo in README.md

0.3.0
-----

* Fixed an anchor
* Language cleanup in README
* README fix
* Updated REAMDE and model command docstring
* Add unit test case for \`ratom model\`
* Add version string validation test
* Fix type hint
* Log message tweaks
* Add stale spaCy model version warning
* Update internal list of spaCy models
* Model list handling WIP
* Model list handling WIP
* Model list handling WIP
* Add unit test around list models utility
* Make DistributionNotFound except work in tox
* Update spaCy model query logic
* Formatting
* Update unit test
* Group spaCy model versions in \`ratom model -l\` output
* Fix unit test
* Expand unit testing for \`ratom model ...\`
* Add post-install model version checks
* Add unit tests for \`ratom model\`
* Update CLI messages and media types file
* Spacy model management WIP - finalize command line interface
* Spacy model management WIP - simplify command line interface
* Add docstring for ratom report
* Update requirements
* Add \`ratom model\` command
* Skip before\_script in release stage
* Fix OSX script invocation
* Fix command help string for OSX
* Fix formatting
* Update CI pre-requisites
* Update CI pre-requisites
* Remove travis\_wait mode
* Add before\_script step
* Add edrm download utility

0.2.6
-----

* Add optional internal tree to MboxArchive
* Update media\_types.json
* Mock failed http requests instead of using an external service
* Remove redundant setting
* Store RATOM\_xxx settings in DB
* Add RATOM prefix to progress step setting
* Update spacy model list (#110)
* Update README.md
* Expand reporting
* Fix message counter in get\_messages()
* Add unit test for ratom report
* Fix imports
* Move some code from libratom.lib.entities to libratom.lib.core
* Use MSG\_PROGRESS\_STEP in get\_messages()
* Update chunk size and commit batch size defaults
* Add processing progess bar
* Use physical core count as default number concurrent jobs
* Add psutil and update requirements
* Update media\_types.json
* Add report subcommand to generate a DB report without any message processing
* DB report only - WIP
* Fix typo
* Use imap\_unordered for processing messages
* Update README.md
* Windows install clarification and general cleanup
* Catch ChunkedEncodingError
* Add basic retry loop against connection errors
* Add connection retry backoff factor
* Add connection retry backoff factor
* Update file download utility code
* Update file download utility code
* Update enron dataset URL to ibiblio.org

0.2.5
-----

* Update dependencies

0.2.4
-----

* Stop testing for specific entity counts
* Update dependencies
* Adapt media type list validation test for coverage

0.2.3
-----

* Formatting
* Fix media type list validation test
* Skip media type list validation test in CI
* Update media type list
* Add codeclimate badge
* Add unit test for attachment media type validation
* Slightly format requirement files
* Updated table descriptions
* Update dependencies
* Update unit test to reflect lower verbosity
* Decrease log level for certain message parsing errors (#94)
* Refactor exception handling in get\_messages() (#94)
* Remove Xcode9.4 with Python3.6 from test matrix
* Add progress update to extract\_entities() (#93)
* Remove progress update from message generator (#93)
* Pass jobs parameter to scan\_files() (#95)
* Add Python 3.8 to CI (#89)
* Add unit test for get\_file\_info()
* Remove dead code
* Remove dead code
* Remove codacy badge (#65)
* Add python 3.6 dependency
* Update unit test
* Sort media type list
* Store attachment info in DB (#84)
* Remove dashes and colons from DB file timestamp (#90)
* Attachment WIP: Use DB session for intermediate file scan results
* Attachment WIP: refactoring
* Attachment WIP: formatting
* Attachment WIP: formatting
* Attachment WIP: media type utility unit test
* Mbox attachment support WIP
* Update test entity counts
* Formatting
* Update dependencies
* Attachment WIP: mbox attachment info extraction
* Attachment WIP: refactoring
* Attachment WIP: refactoring
* Attachment WIP: refactoring
* Attachment WIP: Add get\_attachment\_metadata implementation for PffArchive
* Attachment WIP: Add data files to packaging
* Attachment WIP: Add json list of MIME types
* Attachment WIP: Add external script
* Attachment WIP: Add external script
* Attachment WIP
* Attachment WIP
* Add attachment model (#84)
* Update test entity counts
* Allow lazy import
* Update dependencies
* Test CTRL-C during the file scan phase
* Simulate CTRL-C in entity extraction test

0.2.2
-----

* README.md formatting
* Add pylint to CI tests (#65)
* Handle missing CTE header when exporting an mbox message as string
* Log file path in addition to exception info
* Make unit test more explicit
* Test string conversion from message without CTE header, confirm failure
* Update project summary

0.2.1
-----

* README refinements and clarifications
* README refinements and clarifications
* README refinements and clarifications
* Update unit tests
* Improved install language
* Improved install language
* Improved install language
* Improved install language
* Minor README cleanup
* Clarified install requirements
* Updated schema description
* Markdown consistency fixes
* Markdown consistency fixes
* Environment text fixes
* Markdown fix
* Markdown fix
* Added environment setups
* Add configuration information to CLI commands DB output
* Add OSX with xcode11.2 to test matrix
* Lazy build of PffArchive's internal tree
* Lazy build of PffArchive's internal tree
* Use internal tree representation in PffArchive.get\_message\_by\_id()
* Perform Python 3.7 test/build jobs on Ubuntu 18.04LTS

0.2.0
-----

* Resolved some README typos and language issues
* Updated entities cli explanation text
* Added schema svg
* Added schema svg
* Added flag examples
* Updated README to reflect mbox support
* Reduce chunksize for file report
* Update test entity counts
* Set travis\_wait to 60 minutes
* Use 2 jobs in CI
* Skip long running tests on OSX
* Add unit tests for download\_files() utility
* Update test entity counts and use travis\_wait
* Increase pytest verbosity
* Update test entity count following spaCy upgrade
* Update dependencies
* Use transport adapter with requests
* Add multithreaded download utilities
* Add unit test for unsupported archive type
* Use Requests.Session in mbox files fixture
* Add scope to mbox files fixture
* pst / mbox file type detection
* Adjust unit test assertion to code change
* More edits for mbox support
* Add mbox entity extraction test
* Exclude abstract base class from coverage
* Refactor entity extraction test code
* Only catch ORM exceptions in session context manager
* Fix formatting
* Add basic mbox unit test
* Use generic archive type
* Formalize archive class interface
* mbox support - WIP
* mbox support - WIP

0.1.2
-----

* Update README.md
* Minor update to README.md
* Update DB schema in README.md
* Link ORM objects through relationships instead of foreign keys
* Tweak commit strategy
* Handle empty relationship in FileReport properties (#45)
* Add version badge (#51)
* Add unit test for database reports (#45)
* Add FileReport model properties (#45)
* Add timestamps to message reports (#45)
* Add file checksum step to entity extraction code (#45)
* filename -> filepath (#45)
* Link messages to files (#45)
* Add foreign key relationships between models (#45)
* Add foreign key relationships in models (#45)
* Refactoring (#45)
* Processing reports WIP (#45)
* Processing reports WIP (#45)
* Processing reports WIP (#45)
* Processing reports WIP (#45)
* Add models for file processing report (#45)
* Perform static analysis before test suite
* Add missing type hint
* Add PffArchive.get\_message\_by\_id() method
* Add type hints to PffArchive's public methods
* Updated copyright
* Updated copyright

0.1.1
-----

* Update requirements
* Use pip3 in CI
* Update README.md
* Update setup.cfg
* Fix travis configuration
* Fix travis configuration \* No release stage in PRs \* Run script phase in release stage
* Set up travis build stages
* Remove dead code
* Wrap pip-compile calls in utility script
* Update libpff-python-ratom dependency version
* Update libpff-python-ratom dependency version
* Update requirements
* flake8 fix
* Add unit tests for message generator
* Add unit tests for message generator
* Test entities subcommand
* Break entity extraction code into smaller pieces
* Rename libratom.utils libratom.lib
* Break entity extraction code into smaller pieces - WIP
* Add mock context manager to consolidate code paths
* Fix typo

0.1.0
-----

* Update travis deploy configuration
* Update codecov configuration
* Updated README, entity extraction
* Confirm entity counts in unit test (#24)
* Fix unit test (#30)
* Fix fixture (#30)
* Unit test with large messages (#30)
* Debug message tweak (#24)
* Warning message tweak (#24)
* More user friendly exception handling (#24)
* Make error messages more user friendly (#24)
* Update requirements-dev.txt
* Use the same label for env var/setting (#30)
* Update requirements (#24)
* Remove redundant log message (#24)
* Switch progress bar to enlighten (#24)
* Add env var hook for spacy model max text length (#30)
* Allow for passing progress\_callback=None in optional arguments (#24)
* Pass progress update callback only instead of entire object (#24)
* Add unit test (#29)
* Add file and message info to spacy error message (#29)
* Fix return value of entity extractor upon error (#29)
* Handle bad files during the message count phase (#28)
* Exit cleanly if spaCy cannot install a model (#24)
* Exit cleanly if spaCy cannot install a model (#24)
* Specify spaCy model as runtime option (#24)
* Environment variable hooks for certain module constants (#24)
* DRY (#24)
* Add a unit test (#24)
* Track progress on a per message basis (#24)
* Don't download spacy model prior to running the test suite (#24)
* Prevent spacy's CLI output from interfering with ours (#24)
* Download spacy model if not found (#24)
* Leave partial output file on disk after CTRL+C (#24)
* Add codedov patch configuration
* Remove incomplete DB file as part of CTRL+C handling (#24)
* Add codecov configuration
* CLI help message tweak (#24)
* Pylint tweak (#24)
* Check DB file after CLI test (#24)
* Add test with file source (#24)
* Remove unnecessary validator (#24)
* Add actual entity extraction to unit tests (#24)
* Minor test code refactor (#24)
* Extend CLI unit tests (#24)
* Extend CLI unit tests (#24)
* CLI unit test stub (#24)
* Update requirements-dev.txt
* Add more type hints
* Configure isort so that it doesn't conflict with black
* Add optional progress bar to CLI (#24)
* Handle CTRL+C cleanly (#24)
* Move concurrency code into its own module (#24)
* Minor edits
* Entity extraction with multiprocessing.Pool
* Move code around to accomodate multiprocessing.Pool
* Entity extraction WIP
* isort fix
* Entity extraction WIP
* Entity extraction WIP
* Add sqlalchemy to the requirements
* Move code out of libratom/utils
* isort fix
* Add docstrings
* Update requirements-dev.txt
* Update requirements.txt
* Entities extraction WIP
* black formatting
* isort fix
* Update requirements-dev.txt
* Update requirements.txt
* Finalize CLI
* Small CLI tweaks
* Small CLI tweaks
* Add output path validation
* Update requirements.txt
* command line interface for ratom entities
* command line interface stub
* CLI WIP
* Minor edit of README.md
* Language fix
* README updated with installation info and license
* Update example/README.md
* Update example/README.md
* Remove sample notebooks and add mybinder link
* Remove sample notebooks and add mybinder link
* Make headers optional in PffArchive.format\_message()
* gitignore ChangeLog and AUTHORS
* gitignore Changelog and AUTHORS
* Remove Changelog and AUTHORS from version control
* Disable git --depth flag in travis config
* Update .travis.yml and setup.cfg
* Skip PyPI deployment on OSX
* Add wheel distribution
* Set up PyPI deployment from Travis
* Update requirements
* OSX Travis setup
* Update Travis build matrix
* Add requirements for sample notebooks
* Remove notebook draft
* Clean up entity extraction notebooks
* Rename entity extraction notebooks
* Finalize concurrency notebooks
* concurrency WIP
* concurrency WIP
* concurrency WIP
* Add widget box to show extracted entities (#11)
* Add entity extraction notebook (#11)
* Entities extraction WIP
* Entities extraction WIP
* Clear notebook outputs
* Add notebook
* Pylint tuning
* Extend static analysis to test code
* Remove assertion in load test
* Remove assertion in load test
* Tweak unit tests
* Update dev requirements
* Make pytest verbose in CI
* Strip header string in PffArchive.format\_message
* Pylint tuning
* Add parameter-based load test
* Improve load test
* Formatting
* Add logging to load test
* First pass at load testing
* load testing - WIP
* Upgrade dependencies
* Add dependency
* Update pylint settings
* Add attachment extraction test
* Increase testing of PffArchive
* Formatting
* Update requirements
* Add internal tree structure to PffArchive class
* Try to get rich body if plain text is null/empty (#6)
* Formatting (#6)
* Add test fixture (#6)
* First pass at message serializing (#6)
* Address linter issues (#3)
* DRY (#3)
* DRY (#3)
* Expand unit testing (#3)
* Improve CI setup script (#3)
* Add CI utility (#3)
* Add PffArchive class (#3)
* Add fixture to fetch PST test file (#3)
* Update requirements (#3)
* Add black and isort to static checks
* Update requirements
* Set up CI and packaging
* Add Codacy badge
* Initial commit
