Machine Learning application in Finance Python package¶
MLfin.py is an Advance Machine Learning toolbox for financial applications. The main ideas is using proprietary works and code snippent by Dr. Marcos López de Prado to build a morden Pythonic package that implements newest tech stacks from various libraries such as Numpy, Pandas, Numba, and Scikit-Learn. This work inspired by the library MlFinLab by Hudson and Thames. Unfortunately, the library is closed-source and I believe in the power of open source projects, it motivates me to build this package from ground up.
Leverage best practice in packaging Python library, morden documentation style and comprehensive examples, MLfin.py will be the great tool for Quant Researchers, Algorithmic Traders, and Data Scientists as well as Finance students to reproduce the complex data transformation, labeling, sampling and feature engineering techniques with ease.
Installation¶
Installation can then be done via pip:
pip install mlfinpy
For the sake of best practice, it is good to do this with a dependency manager. I suggest you set yourself up with poetry, then within a new poetry project run:
poetry add mlfinpy
Note
If any of these methods don’t work, please raise an issue with the packaging label on GitHub.
For developers¶
If you are planning on using Mlfinpy as a starting template for significant modifications, it probably makes sense to clone the repository and to just use the source code:
git clone https://github.com/baobach/mlfinpy
Alternatively, if you still want the convenience of a global from mlfinpy import x,
you should try:
pip install -e git+https://github.com/baobach/mlfinpy.git
Work with HFT Data¶
In reality, testing code snippets through the first 3 chapters of the book is challenging as it relies on HFT data to create the new financial data structures. Sourcing the HFT data is very difficult and thus TickData LLC provides the full history of S&P500 Emini futures tick data and available for purchase.
I am not affiliated with TickData in any way but would like to recommend others to make use of their service. The full history costs $750 and is worth every penny. They have really done a great job at cleaning the data and providing it in a user friendly manner.
Download Sources¶
TickData does offer about 20 days worth of raw tick data which can be sourced from their website link. For those of you interested in working with a two years of sample tick, volume, and dollar bars, it is provided for in the research repo. You should be able to work on a few implementations of the code with this set.
Note
Searching for free tick data can be a challenging task. The following three sources may help:
Dukascopy. Offers free historical tick data for some futures, though you do have to register.
Most crypto exchanges offer tick data but not historical (see Binance API). So you’d have to run a script for a few days.
Blog Post: How and why I got 75Gb of free foreign exchange “Tick” data.
Datasets¶
To make the developing module and testing the code process more convenient, MLfin.py package contains various financial datasets which can be used by a developer as sandbox data.
Tick Data Sample¶
MLfin.py provides a sample of tick data for E-Mini S&P 500 futures which can be used to test bar compression algorithms,
microstructural features, etc. Tick data sample consists of Timestamp, Price and Volume. The data contain
500,000 rows of cleaned tick data.
Dollar-Bar Data Sample¶
We also provide a sample of dollar bars for E-Mini S&P 500 futures. Data set structure:
Open price (open)
High price (high)
Low price (low)
Close price (close)
Volume (cum_volume)
Dollar volume traded (cum_dollar)
Number of ticks inside of bar (cum_ticks)
Tip
You can find more information on dollar bars and other bar compression algorithms in Financial Data Structures.
ETF Prices Sample¶
- mlfinpy.dataset.load_datasets.load_stock_prices() DataFrame[source]¶
Loads stock prices data sets consisting of EEM, EWG, TIP, EWJ, EFA, IEF, EWQ, EWU, XLB, XLE, XLF, LQD, XLK, XLU, EPP, FXI, VGK, VPL, SPY, TLT, BND, CSJ, DIA starting from 2008 till 2016.
Returns¶
- pd.DataFrame
The stock_prices data frame.
The data set consists of close prices for:
EEM, EWG, TIP, EWJ, EFA, IEF, EWQ, EWU, XLB, XLE, XLF, LQD, XLK, XLU, EPP,FXI, VGK, VPL, SPY, TLT, BND, CSJ, DIA
Starting from 2008 till 2016.
It can be used to test and validate portfolio optimization techniques.
Example¶
from mlfinpy.datasets import (load_tick_sample, load_stock_prices, load_dollar_bar_sample)
# Load sample tick data
tick_df = load_tick_sample()
# Load sample dollar bar data
dollar_bars_df = load_dollar_bar_sample()
# Load sample stock prices data
stock_prices_df = load_stock_prices()
Contents¶
Other information
Project principles and design decisions¶
It should be easy to swap out individual components of each module with the user’s proprietary improvements.
Usability is everything: it is better to be self-explanatory than consistent.
The goal is creating a framework to build a robust and functional library for machine learning applications.
Everything that has been implemented should be tested and formatted with lattest requirements.
Inline documentation is good: dedicated (separate) documentation is better. The two are not mutually exclusive.
Formatting should never get in the way of good code: because of this, I have deferred all formatting decisions to Black, Flake8, and Isort.