Skip to content

python_setup

guoling edited this page Jul 5, 2024 · 13 revisions

MMKV for Python (on POSIX & Windows)

MMKV is an efficient, small, easy-to-use mobile key-value storage framework used in the WeChat application. It's currently available on both Android, iOS/macOS, Windows and Python/Golang (POSIX).

Getting Started

Prerequisites

  • Python 2.7 and above.
  • CMake 3.8.0 and above.
  • C++ compiler that supports C++ 20 standard.
  • Linux(Ubuntu, Arch Linux, CentOS, Gentoo), Unix(macOS, FreeBSD, OpenBSD) & Windows are supported.

Installation Via Source

  1. Getting source code from the git repository:
git clone https://github.com/Tencent/MMKV.git
  1. Prepare pybind11.
    2.1 Get pybind11 (v2.5.0) source code:
cd MMKV
git submodule update --init --recursive

2.2 On Linux, install python-dev for Python 2.x, or python3-dev for Python 3.x. On macOS or Windows, you don't need to install those things. See also pybind11 docs.
For example, let's say we are using Python 3.x on Ubuntu:

sudo apt-get install python3-dev
  1. Build & install the MMKV native library for Python (you might need root permission to install it). Assuming we are using Python 3.x:
cd ./Python
python3 setup.py install
# or if you're not login as root
sudo python3 setup.py install

That's it. We are all set.

  • Note: If you don't want to install MMKV, you can just build the native library:
cd ./Python
cmake -B build
cmake --build build --config Release -j 8

You can find out which Python is used to build MMKV from the logs of cmake. It's something like:

-- Found PythonInterp: /usr/bin/python3.6 (found version "3.6.9") .

And if you find it not correct (let's say you want to use Python 2.7 instead), you can surpass it by setting PYBIND11_PYTHON_VERSION for cmake:

cmake -DPYBIND11_PYTHON_VERSION=2.7 ..

Then you can copy the generated mmkv dylib (the exact name is vary on different platforms, for example mmkv.cpython-36m-x86_64-linux-gnu.so or mmkv.lib on Windows) to your Python library directory PYTHONPATH. If you are using Python 3, you can just copy it to your code directory.

  1. Test MMKV:
python3 demo.py
python3 unit_test.py

Tutorial

You can use MMKV as you go. All changes are saved immediately, no save, no sync calls are needed.

Configuration

  • Setup MMKV on App startup, say in your main() function, add these code:

    import mmkv
    
    if __name__ == '__main__':
        mmkv.MMKV.initializeMMKV(tempfile.gettempdir() + '/mmkv')

CRUD Operations

  • MMKV has a global instance, that can be used directly:

    kv = mmkv.MMKV.defaultMMKV()
    
    kv.set(True, 'bool')
    print('bool = ', kv.getBool('bool'))
    
    kv.set(-1 * (2 ** 31), 'int32')
    print('int32 = ', kv.getInt('int32'))
    
    kv.set((2 ** 32) - 1, 'uint32')
    print('uint32 = ', kv.getUInt('uint32'))
    
    kv.set(2 ** 63, 'int64')
    print('int64 = ', kv.getLongInt('int64'))
    
    kv.set((2 ** 64) - 1, 'uint64')
    print('uint64 = ', kv.getLongUInt('uint64'))
    
    kv.set(3.1415926, 'float')
    print('float = ', kv.getFloat('float'))
    
    kv.set('Hello world, MMKV for Python!', 'string')
    print('string = ', kv.getString('string'))
    
    lst = range(0, 10)
    kv.set(bytes(lst), 'bytes')
    bt = kv.getBytes('bytes')
    print('raw bytes = ', bt, ', decoded bytes = ', list(bt))

    As you can see, MMKV is quite easy to use.

  • Deleting & Querying:

    kv = mmkv.MMKV.defaultMMKV()
    print('keys before remove:', sorted(kv.keys()))
    
    kv.remove('bool')
    print('"bool" exist after remove: ', ('bool' in kv))
    
    kv.remove(['int32', 'float'])
    print('keys after remove:', sorted(kv.keys()))
  • If different modules/logics need isolated storage, you can also create your own MMKV instance separately:

    kv = mmkv.MMKV('test_python')
    kv.set(True, 'bool')
  • If multi-process accessing is needed, you can set MMKV_MULTI_PROCESS on MMKV initialization:

    kv = mmkv.MMKV('test_python', mmkv.MMKVMode.MultiProcess)
    kv.set(True, 'bool')

Supported Types

  • Primitive Types:

    • Boolean, int, float

    Note: Due to the limitation of protobuf, int32, uint32, int64, uint64 has a different encoding & decoding protocol. You should call different getXXXInt() method according to the value's possible range. For each range of those integer types, checkout MMKV/Python/unit_test.py.

  • Classes:

    • str, bytes

Logs

  • By default, MMKV doesn't print any logs. You can turn on logging in initialization:

    mmkv.MMKV.initializeMMKV(tempfile.gettempdir() + '/mmkv', mmkv.MMKVLogLevel.Info)
  • MMKV prints log to stdout, which is not convenient for the interpreter or diagnosing online issues. You can set up MMKV log redirecting on App startup. Implement a callback function with signature (logLevel: mmkv.MMKVLogLevel, file: str, line: int, function: str, message: str) -> None, register it as log handler. Remember to unregister it before exiting, otherwise, your script won't exit properly.

def logger(log_level, file, line, function, message):
    level = {
        mmkv.MMKVLogLevel.NoLog : 'N',
        mmkv.MMKVLogLevel.Debug : 'D',
        mmkv.MMKVLogLevel.Info : 'I',
        mmkv.MMKVLogLevel.Warning : 'W',
        mmkv.MMKVLogLevel.Error : 'E'
    }
    # use your logging tools instead of print()
    print('[{0}] <{1}:{2}:{3}> {4}'.format(level[log_level], file, line, function, message))


if __name__ == '__main__':
    # enable logging & redirect logging
    mmkv.MMKV.initializeMMKV(tempfile.gettempdir() + '/mmkv', mmkv.MMKVLogLevel.Info, logger)
    
    # some logic
    ...

    # unregister before exit, otherwise the script won't exit properly
    mmkv.MMKV.unRegisterLogHandler()

    # or just call onExit() will do the job
    # mmkv.MMKV.onExit()

Backup & Restore

  • You can use MMKV's backup & restore API to backup your data to somewhere else, and restore them later.

      root_dir = tempfile.gettempdir() + '/mmkv_backup'
      // backup one instance
      ret = mmkv.MMKV.backupOneToDirectory(mmap_id, root_dir)
      // backup all instances
      count = mmkv.MMKV.backupAllToDirectory(root_dir)
    
      // restore one instance
      ret = mmkv.MMKV.restoreOneFromDirectory(mmap_id, root_dir)
      // restore all instances
      count = mmkv.MMKV.restoreAllFromDirectory(root_dir)

Auto Expiration

  • Starting from v1.3.0, you can upgrade MMKV to auto key expiration. Note that this is a breaking change. Once upgraded to auto key expiration, the file is not valid for any older version of MMKV (<= v1.2.16) to operate correctly.

  • Global Expiration. The most simple way to do it is to turn on auto key expiration for all keys in the whole file.

    // expire in a day
    kv.enableAutoKeyExpire(24 * 60 * 60);

    Or, if you prefer, you can enable auto key expiration without setting a global expiration duration. In this case, each key is not expired by default.

    // enable auto key expiration without global duration
    kv.enableAutoKeyExpire(0);
  • Individual Expiration. You can set a special expiration duration for a key, regardless with the file has a global duration or not. Note that you have to enable auto key expiration first.

    // enable auto key expiration with an hour duration
    kv.enableAutoKeyExpire(60 * 60);
    
    // set a key with the file's global expiration duration, aka 60 * 60
    kv.set("some value", "key_1");
    
    // set a special key that expires in two hours
    kv.set("some value", "key_2", 2 * 60 * 60);
    
    // set a special key that never expires
    kv.set("some value", "key_3", 0);

    Or, if you prefer, you can enable auto key expiration without setting a global expiration duration. In this case, each key is not expired by default.

    // enable auto key expiration without global duration
    kv.enableAutoKeyExpire(0);
    
    // set a key that never expires
    kv.set("some value", "key_1");
    
    // set a special key that expires in an hour
    kv.set("some value", "key_2", 60 * 60);
  • The expire duration is counted in seconds. You can use any other duration you prefer. For example, expiration in a week is 7 * 24 * 60 * 60.