A key-value store is a simple yet powerful type of database that allows you to associate unique keys with corresponding values.

While it is conceptually similar to a Python dictionary, a key-value store can be enhanced to support more advanced features like persistence, expiration, and encryption.

In this article, we’ll walk through how to create a key-value store in Python, starting with an in-memory version and progressing to more advanced implementations.

By the end, you'll have the tools and knowledge to create your own key-value store tailored to your application's specific needs.

What is a Key-Value Store?

A key-value store allows you to map keys to specific values, providing fast lookups, inserts, and updates.

These systems are used in applications like caching, configuration storage, and even large distributed databases like Redis or DynamoDB.

This is the simplest form of a key-value store and serves as a strong foundation for more advanced implementations.

class InMemoryKeyValueStore:  

    def __init__(self):  

        """Initialize an empty dictionary to store key-value pairs."""  

        self.store = {}  



    def set(self, key, value):  

        """Store a key-value pair."""  

        self.store[key] = value  



    def get(self, key):  

        """Retrieve the value for a given key."""  

        return self.store.get(key)  



    def delete(self, key):  

        """Remove a key-value pair."""  

        if key in self.store:  

            del self.store[key]  



    def keys(self):  

        """Return a list of all keys."""  

        return list(self.store.keys())  





if __name__ == '__main__':  

    # Usage example  

    store = InMemoryKeyValueStore()  

    store.set('name', 'Alice')  

    print(store.get('name'))  # Output: Alice  

    store.delete('name')  

    print(store.get('name'))  # Output: None

This code defines a simple in-memory key-value store implemented as a Python class named InMemoryKeyValueStore.

The class initializes an empty dictionary to store key-value pairs and provides methods to interact with this store:

• The __init__ method sets up the dictionary.


• The set method allows adding or updating key-value pairs.


• The get method retrieves values associated with specific keys, returning None if the key does not exist.


• The delete method removes key-value pairs from the store if the key is present.


• The keys method returns a list of all keys currently stored in the dictionary.

The usage example demonstrates how to create an instance of the InMemoryKeyValueStore class, set a key-value pair, retrieve the value, delete the key, and attempt to retrieve the value again after deletion.

This simple approach is useful for applications that need a temporary, fast key-value store with minimal overhead.

Create a File-Based Key-Value Store

To make the store persistent, we’ll save key-value pairs to a file. We’ll use the json module to serialize and deserialize data.

This is useful for caching, where you want to keep data only for a limited time.

import time  

from InMemory import InMemoryKeyValueStore  





class ExpiringKeyValueStore(InMemoryKeyValueStore):  

    def __init__(self):  

        super().__init__()  

        self.expiration_times = {}  



    def set(self, key, value, ttl=None):  

        super().set(key, value)  

        if ttl is not None:  

            self.expiration_times[key] = time.time() + ttl  



    def get(self, key):  

        if key in self.expiration_times and time.time() > self.expiration_times[key]:  

            self.delete(key)  

        return super().get(key)  





if __name__ == '__main__':  

    # Usage example  

    store = ExpiringKeyValueStore()  

    store.set('name', 'Alice', 1)  

    print(store.get('name'))  # Output: Alice  

    time.sleep(2)  

    print(store.get('name'))  # Output: None

This code defines an expiring key-value store implemented as a Python class named ExpiringKeyValueStore.

The class extends the functionality of an in-memory key-value store by adding support for time-to-live (TTL) expiration of keys.

This means that keys can be set to automatically expire after a specified amount of time:

• The ExpiringKeyValueStore class inherits from InMemoryKeyValueStore, which provides basic key-value store functionality.


• The __init__ method initializes the class by calling the parent class's initializer and setting up an additional dictionary, expiration_times, to store the expiration times for keys.


• The set method extends the parent class's set method to include an optional ttl (time-to-live) parameter. If ttl is provided, the method calculates the expiration time (current time plus ttl) and stores it in the expiration_times dictionary.


• The get method checks if a key has expired by comparing the current time with the key's expiration time. If the key has expired, it is deleted from the store. The method then retrieves the value using the parent class's get method.

The usage example demonstrates how to create an instance of the ExpiringKeyValueStore class, set a key-value pair with a TTL, retrieve the value, and observe the expiration of the key:

• An instance of ExpiringKeyValueStore is created.


• The set method is used to store the key-value pair ('name', 'Alice') with a TTL of 1 second.


• The get method is used to retrieve the value associated with the key 'name', which outputs Alice.


• The program sleeps for 2 seconds to allow the TTL to expire.


• The get method is used again to retrieve the value associated with the key 'name', which outputs None since the key has expired and been deleted.

This example showcases the functionality of the expiring key-value store, including setting keys with a TTL, retrieving values, and handling key expiration.

💡 Keys with a TTL will be automatically deleted when expired, which is useful for caching frequently changing data.

Encryption

To protect sensitive data, you can encrypt stored values using the cryptography library.

It uses a persistent dictionary-like object backed by a database file, allowing for efficient storage and retrieval of data without loading the entire dataset into memory.

import shelve  





class ShelveKeyValueStore:  

    def __init__(self, filename='shelve_store.db'):  

        self.filename = filename  



    def set(self, key, value):  

        with shelve.open(self.filename) as db:  

            db[key] = value  



    def get(self, key):  

        with shelve.open(self.filename) as db:  

            return db.get(key)  



    def delete(self, key):  

        with shelve.open(self.filename) as db:  

            if key in db:  

                del db[key]  



    def keys(self):  

        with shelve.open(self.filename) as db:  

            return list(db.keys())  





# Usage example  

if __name__ == '__main__':  

    # Usage example (first run)  

    store = ShelveKeyValueStore()  

    store.set('language', 'Python')  

    print(store.get('language'))  # Output: Python  



    # Usage example (second run)    

    store = ShelveKeyValueStore()  

    print(store.get('language'))  # Output: Python  

    store.delete('language')  



    # Usage example (third run)  

    store = ShelveKeyValueStore()  

    print(store.get('language'))  # Output: None

This code defines a key-value store implemented as a Python class named ShelveKeyValueStore.

The class uses the shelve module to persist key-value pairs in a database file, allowing data to be retained across different runs of the program.

The class provides methods to set, get, delete, and list keys, ensuring that the data is securely stored and can be accessed as needed:

• The ShelveKeyValueStore class initializes with a specified filename (defaulting to 'shelve_store.db').


• The set method stores a key-value pair in the database.


• The get method retrieves the value associated with a specific key, returning None if the key does not exist.


• The delete method removes a key-value pair from the database if the key is present.


• The keys method returns a list of all keys currently stored in the database.

The usage example demonstrates how to create an instance of the ShelveKeyValueStore class, set a key-value pair, retrieve the value, delete the key, and observe the persistence of data across multiple runs of the program:

• In the first run, an instance of ShelveKeyValueStore is created, a key-value pair ('language', 'Python') is set, and the value is retrieved and printed, outputting Python.


• In the second run, a new instance is created, the value for the key 'language' is retrieved and printed (showing persistence), outputting Python, and the key is then deleted.


• In the third run, another instance is created, and the value for the key 'language' is retrieved and printed, outputting None since the key was deleted in the previous run.

This example showcases the functionality of the shelve-based key-value store, including setting, getting, deleting keys, and the persistence of data across different runs of the program.

💡 This approach is similar to the file-based one but reduces memory usage, making it more efficient for large datasets.

Conclusion

We’ve explored the process of creating a key-value store in Python, starting with simple in-memory stores and progressing to more advanced implementations that include file storage, expiration, encryption, and memory optimization.

Each step builds on the previous one, adding layers of functionality and complexity to meet various application needs.

• 
  In-Memory Stores: These are the simplest form of key-value stores, using Python dictionaries to hold data in memory. They are fast and efficient for small datasets but lack persistence, meaning data is lost when the program terminates.


• 
  File-Based Stores: These stores use files to persist data, ensuring that information is retained across program runs, making them suitable for applications requiring data persistence.


• 
  Expiring Key-Value Stores: Adding a time-to-live (TTL) feature allows keys to expire after a specified duration. This is crucial for applications like caching, where data relevance is time-sensitive.


• 
  Encrypted Key-Value Stores: Incorporating encryption ensures that stored data is secure. This is essential for applications handling sensitive information, such as user credentials or financial data.


• 
  Memory Optimization: Techniques like using efficient data structures and managing memory allocation can optimize the performance of key-value stores, making them suitable for large-scale applications. For example, the shelve module in Python is a memory-optimized solution for key-value storage

These concepts are essential for applications such as caching, session management, and configuration storage.

Understanding and implementing these various types of key-value stores equips developers with the tools to build robust, efficient, and secure data storage solutions tailored to specific application requirements.

Follow me on Twitter: 

Follow me on TikTok: