Skip to content

muni106/linux-caching-system

Folders and files

NameName
Last commit message
Last commit date

Latest commit

Β 

History

5 Commits
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 

Repository files navigation

πŸš€ Linux Caching System

License: MIT Ubuntu Vagrant WireGuard

A high-performance, secure caching infrastructure for APT packages in Debian-based environments

Developed as part of my thesis project at the University of Bologna (2023/2024), this system optimizes package management in corporate environments through intelligent caching and secure VPN connectivity.


πŸ“‹ Table of Contents


🎯 Overview

In modern corporate environments, managing software packages across multiple machines can be challenging. Repeatedly downloading the same packages from remote repositories leads to:

  • ⚠️ Excessive bandwidth consumption
  • 🐌 Slow download speeds
  • πŸ’Έ Increased operational costs
  • πŸ”„ Network congestion

This project addresses these challenges by implementing a centralized caching proxy that stores downloaded packages locally and serves them to multiple clients through a secure VPN tunnel.


✨ Key Features

🎯 Performance

  • Smart Caching: First download caches the package; subsequent requests served instantly
  • Bandwidth Optimization: Reduces redundant downloads by up to 80%
  • Speed Improvement: Significantly faster package installation across the network

πŸ”’ Security

  • WireGuard VPN: Modern, lightweight VPN with state-of-the-art cryptography
  • Encrypted Communications: All traffic between clients and server is protected
  • Network Isolation: Secure tunnel prevents unauthorized access

πŸ› οΈ Infrastructure

  • Automated Deployment: Complete setup via Vagrant provisioning scripts
  • Reproducible Environments: Consistent configuration across all instances
  • Scalable Architecture: Easy to add new clients to the network

πŸ—οΈ Architecture

β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
β”‚                    Corporate Network                    β”‚
β”‚                                                         β”‚
β”‚  β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”         β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”            β”‚
β”‚  β”‚   Client 1   β”‚         β”‚   Client 2   β”‚            β”‚
β”‚  β”‚  (Ubuntu)    β”‚         β”‚  (Ubuntu)    β”‚            β”‚
β”‚  β””β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”˜         β””β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”˜            β”‚
β”‚         β”‚                        β”‚                     β”‚
β”‚         β”‚   WireGuard VPN        β”‚                     β”‚
β”‚         β””β”€β”€β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜                     β”‚
β”‚                  β”‚                                      β”‚
β”‚         β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β–Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”                           β”‚
β”‚         β”‚  Cache Server    β”‚                           β”‚
β”‚         β”‚  apt-cacher-ng   β”‚                           β”‚
β”‚         β”‚  + WireGuard     β”‚                           β”‚
β”‚         β””β”€β”€β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜                           β”‚
β”‚                  β”‚                                      β”‚
β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜
                   β”‚
                   β–Ό
          Internet / APT Repositories

Component Overview

Component Technology Purpose
Virtualization Vagrant + VirtualBox Automated VM management
Cache Server apt-cacher-ng Package caching proxy
VPN WireGuard Secure client-server communication
OS Ubuntu 18.04 (Bionic) Base system for all machines

πŸ’» Technologies

Core Stack

Why These Technologies?

Vagrant over Docker/Multipass:

  • Complete OS-level isolation
  • Better network control for VPN configuration
  • Easier filesystem management for caching
  • Reproducible environments with simple configuration

apt-cacher-ng over Squid/Approx:

  • Optimized specifically for APT packages
  • Native support for package signatures and dependencies
  • Automatic cache management
  • Simpler configuration for Debian-based systems

WireGuard over OpenVPN/IPsec:

  • Superior performance (lower latency, faster speeds)
  • Modern cryptography
  • Minimal configuration complexity
  • Smaller codebase = reduced attack surface

πŸš€ Getting Started

Prerequisites

Ensure you have the following installed:

Installation

  1. Clone the repository

    git clone https://github.com/yourusername/linux-package-caching.git
    cd linux-package-caching
  2. Configure the environment

    Edit the Vagrantfile if you need to customize:

    • Number of clients
    • Network configuration
    • Resource allocation (RAM, CPU)
  3. Deploy the infrastructure

    vagrant up

    This command will:

    • Create and configure the cache server
    • Set up WireGuard VPN
    • Deploy client VMs
    • Run all provisioning scripts automatically
  4. Verify the setup

    vagrant status

βš™οΈ Configuration

Server Configuration

The server is automatically configured via setup_apt_cacher.sh:

  • apt-cacher-ng: Listens on VPN interface (10.0.0.1:3142)
  • WireGuard: Creates secure tunnel on UDP port 51820
  • Firewall: Configured with iptables for proper routing

Client Configuration

Clients are provisioned using setup_client.sh:

# APT proxy configuration is automatically set to:
Acquire::http::Proxy "http://10.0.0.1:3142";

Adding New Clients

To connect external Debian-based machines:

  1. Copy the client setup script

    scp setup_client.sh user@external-machine:~/
  2. Update the server endpoint in the script:

    SERVER_ENDPOINT="<server-ip>:51820"
  3. Run the setup

    chmod +x setup_client.sh
    ./setup_client.sh
  4. Add peer to server (/etc/wireguard/wg0.conf):

    [Peer]
    PublicKey = <client-public-key>
    AllowedIPs = 10.0.0.X/32
  5. Restart WireGuard on server:

    sudo wg-quick down wg0
    sudo wg-quick up wg0

πŸ“Š Results

The implementation demonstrated significant improvements:

Performance Metrics

  • βœ… Bandwidth Reduction: ~80% decrease in external bandwidth usage
  • βœ… Speed Improvement: 3-5x faster package installation for cached packages
  • βœ… Network Efficiency: Reduced load on upstream repositories

Security

  • πŸ”’ All client-server communications encrypted via WireGuard
  • πŸ”’ Zero-trust network architecture
  • πŸ”’ Protected against man-in-the-middle attacks

Operational Benefits

  • πŸ”„ Fully automated deployment and configuration
  • πŸ“¦ Reproducible infrastructure as code
  • 🎯 Easy scalability for additional clients

πŸ§ͺ Testing

Test with Vagrant Clients

  1. SSH into a client

    vagrant ssh client-1
  2. Update package lists

    sudo apt update
  3. Install a package

    sudo apt install -y htop
  4. Verify caching (install same package on client-2 - should be much faster)

    vagrant ssh client-2
    sudo apt install -y htop

Monitor Cache Performance

Access the apt-cacher-ng web interface:

http://10.0.0.1:3142/acng-report.html

🀝 Contributing

Contributions are welcome! Here's how you can help:

  1. Fork the repository
  2. Create a feature branch (git checkout -b feature/AmazingFeature)
  3. Commit your changes (git commit -m 'Add some AmazingFeature')
  4. Push to the branch (git push origin feature/AmazingFeature)
  5. Open a Pull Request

Ideas for Contributions

  • Support for additional Linux distributions (Fedora, CentOS)
  • Docker-based implementation
  • Monitoring and metrics dashboard
  • Automated testing suite
  • Cloud deployment guides (AWS, Azure, GCP)

πŸ“ License

This project is licensed under the MIT License - see the LICENSE file for details.


πŸ‘¨β€πŸ’» Author

Samite Mounir

Bachelor's Thesis in Computer Science and Engineering
Alma Mater Studiorum - UniversitΓ  di Bologna
Academic Year 2023/2024

Advisor: Prof. Vittorio Ghini


πŸ™ Acknowledgments

Special thanks to:

  • Prof. Vittorio Ghini for supervision and guidance
  • The University of Bologna, School of Engineering
  • The open-source community behind Vagrant, WireGuard, and apt-cacher-ng

πŸ“š Additional Resources


⭐ If you find this project useful, please consider giving it a star! ⭐

Made with ❀️ for the DevOps and Linux

About

No description, website, or topics provided.

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages