How to use Lore: a complete guide to version control for large-scale projects

Introduction to Lore: The Version Control System for Large-Scale Projects

Lore is an open-source version control system developed by Epic Games, specifically designed for large projects that combine source code with large binary assets, such as games and entertainment production workflows.

Traditionally, version control systems like Git struggle to manage very large repositories containing numerous binary files. Game development projects, for example, combine source code with textures, models, audio, video, and other large assets. These files differ from textual source code, making storage, synchronization, and branching inefficient as teams and repositories grow.

Lore solves these problems with a centralized and content-addressed approach. The system stores repository data via content hashes and represents the repository state using Merkle trees and an immutable chain of revisions. This allows data reuse across history and branches while providing integrity checks and a verifiable source of truth.

• Chunked storage: Large files are split into reusable pieces instead of duplicating the entire file with each modification, reducing storage requirements and improving transfer efficiency.
• On-demand hydration and sparse workspaces: Files are retrieved only when needed, avoiding the need to download all repository files initially.
• Lightweight branches and fast switching: Branches are implemented as mutable references rather than full copies of the data, offering full functionality via command-line interface and APIs for integration with various programming languages.

Lore is already integrated as the default version control system for UEFN (Unreal Editor for Fortnite), but currently the open-source implementation cannot communicate with the UEFN version due to a proprietary compression format.

The project is available on GitHub under the MIT license and primarily written in Rust. The current version is 0.8.3, indicating that it is a pre-1.0 release and not yet ready for production.

Prerequisites

To use Lore, ensure you have the following hardware and software requirements:

• Operating System: Linux
• Shell Installer: Available here to start Lore in demo mode
• Project Repository: Available on GitHub under the MIT license
• Programming Language: Rust (version 0.8.3)
• Supported APIs: C/C++, C#, Rust, Go, Python, JavaScript

PROCEDURE: Setting Up Lore on Linux

By the end of this guide, you will have set up Lore on a Linux system in demo mode, creating a local server to test the system.

• Step 1: Access the Quick Start Tutorial
• Step 2: Run the Installation Script

  Open a terminal and enter the following command to download and run the installation script:

  curl -sSf https://raw.githubusercontent.com/EpicGames/lore/main/scripts/install.sh | sh

• Step 3: Start Lore in Demo Mode

  lore demo

• Step 4: Create a Local Server

Verification and Troubleshooting

By the end of this guide, you will be able to test the correct functioning of Lore and resolve any configuration or execution issues.

How to Test if It Works

What to Do if It Fails

Educational Summary and Call to Action

By the end of this guide, you will understand how Lore represents an innovative solution for managing large-scale projects that combine source code and large binary assets, such as video games and production workflows. Lore addresses the limitations of traditional version control systems, offering efficiency in storage, synchronization, and branching.

• Educational Summary: Lore uses a centralized, content-addressed system with chunked storage for large files, allowing them to be split into reusable pieces. It supports on-demand hydration and sparse workspaces, reducing the need to download all files initially. Features are accessible via a CLI and APIs for various programming languages.
• Call to Action: To experiment with Lore, follow the quick start guide to set up a local server in demo mode. Remember that Lore is still in its early stages and is aimed at teams with very large repositories and complex assets. For standard projects, Git remains the best choice.