Save and access data¶
You can save your data persistently in files in the file system. In the Python standard library, there are also several modules for converting data into a linear form. This process is called serialisation or marshalling. The reverse process is called deserialisation or unmarshalling. And if the built-in modules are not sufficient, you can also use the pandas IO tools.
- the marshal module
is mainly used internally by Python and should not be used to store data in a backwards compatible way.
- the pickle module
if you don’t need a readable format or interoperability.
- the json module
you can use to exchange data for different languages in a readable form.
- the xml module
you can also use to exchange data in different languages in a readable form.
Tip
The Python Database API¶
The Python Database API defines a standard interface for Python database access modules. It’s defined in PEP 249 and widely used, for example by sqlite, psycopg, and mysql-python.
SQLAlchemy¶
SQLAlchemy is a widely used database toolkit. It provides not only an ORM but also a generalised API for writing database-agnostic code without SQL. Alembic is based on SQLAlchemy and serves as a database migration tool.
NoSQL databases¶
There is data that is difficult to transfer into a relational data model. At the least then you should take a look at NoSQL databases.