What is a Data Lake Transaction Log?

Last updated Nov 18, 2022 - Edit Source

• Data engineering

The transaction log is the ordered record of every transaction, with a configurable duration that can optionally be set to retain all transactions (i.e. data infinite). A transaction log is a common component used through many of its above-mentioned features, including  ACID Transactions, scalable metadata handling, and  Time Travel. For example,  Delta Lake creates a single  folder called _delta_log.