Update PyArrow version required for PyMongoArrow to use with Fireducks
Update PyArrow version to use with Fireducks - right now I want to use these three libraries in one data science project:
- fireducks-1.2.2 (pip install fireducks)
- pyarrow-19.0.1 (pip install pyarrow) <-- required by the Fireducks
- pymongoarrow-1.6.4 (pip install pymongoarrow) <-- it requires pyarrow 18.0.0
My pipeline connects to MongoDB through PyMongoArrow, downloads big DataFrames and preprocess them with Fireducks superb optimized vector transformations. I tried to side-step the need for PyMongoArrow with code like the attached function, but the end result differs from just using the PyMongoArrow.
What would be needed to update the PyMongoArrow requirements?
4
votes
