Persistent Memory Programming Blog

Here’s a list of blog posts, shown most recent first:


July 10, 2017 – Using Standard Library Containers with Persistent Memory
Introduction Somewhere along the road, when we were doing the C++ bindings for NVML, we found the need for some kind of containers. We were faced with two viable solutions: write everything from scratch or adapt an existing implementation. The obvious choice was NOT to implement from scratch. We would…


July 04, 2017 – Progress Report Q2 2017
It’s summer already (at least in my hemisphere) ! :) And it’s time for the next progress report. Control interface After a very long in the oven, we’ve finally finalized and merged the CTL API which allows for introspection and modification of the internal state of the library. This feature…


April 03, 2017 – Apache Kudu Persistent Memory Enabled Block Cache
Using NVM Libraries To Add Persistent Memory Support to Apache Kudu Block Cache An early project done with the NVM libraries was adding persistent memory support, both volatile and persistent mode, into the Apache Kudu storage engine block cache. This project required modification of existing code. Apache Kudu: http://github.com/apache/kudu My…


March 29, 2017 – Progress Report Q1 2017
It’s been three months already since the last time I wrote something ;) Time really flies by quickly when you are doing interesting stuff. We’ve been very busy with lots of improvements to the library as well as A LOT of tiny fixes: over the last three months the team…


March 09, 2017 – Zero-copy leaf splits in pmemkv
In a B+ tree, splitting a full leaf into two leaves is one of its slowest operations, but pmemkv optimizes this using a zero-copy strategy. Rather then copying any key/value data between full and new leaf, pmemkv splits leaves by swapping persistent structures in place. This minimizes write amplification and…


February 21, 2017 – Introducing pmemkv
We’ve blogged before about building and optimizing key-value stores for persistent memory, and we’re excited to put these ideas to the test in a more formal way. Our new pmemkv project is an open-source key-value store that is optimized for read-heavy workloads on persistent memory. Compared with key-value stores based…


January 25, 2017 – What’s coming in libpmemobj
In my last post I’ve made a promise to share our plans for the near future. So here it is: 4 ideas that we are planning to ship with the upcoming version of libpmemobj. Please note that most of our plans related to libpmemobj are available on our github issues…


January 23, 2017 – Modeling strings with libpmemobj C++ bindings
C++ developers using libpmemobj have more than one option for modeling strings, depending on the size of the strings and whether they are fixed or varying in length. In this post we’ll review the representations that work, known variations to avoid, and finally present a persistent string class that implements…


December 21, 2016 – NVML for Windows
Throughout 2016 a team of engineers from Microsoft, Intel, HPE and HPI have been working to port the NVML project to Windows and we are happy to announce that Technical Preview release is now available! Our main goal in porting the library was to make it as easy as possible…


December 20, 2016 – libpmemobj - a year in review
It’s been a while since the last post on our blog, but we’ve been busy with the recently released 1.2 version of the library. It comes packed with improvements all throughout the code base and it also brings a handful of new features that we hope will end up being…


June 02, 2016 – C++ bindings for libpmemobj (epilogue) - converting existing applications
During the development of the C++ bindings, we wrote a couple of examples and even more tests. But these are new applications written from scratch to understand persistence. While this approach is OK for newly developed apps, there is a lot of existing code out there that is not designed…


May 31, 2016 – C++ bindings for libpmemobj (part 7) - synchronization primitives
To finish off the C++ bindings to libpmemobj blog marathon, I will introduce to you the synchronization mechanisms we implemented. They are mostly C++11-like implementations of different kinds of mutexes and the condition variable. They satisfy their respective concepts (Mutex, SharedMutex and so on), the difference is that they are…


May 25, 2016 – C++ bindings for libpmemobj (part 6) - transactions
As I mentioned in my previous blog post, transactions are the heart of libpmemobj. That is why we had to take utmost care while designing their C++ versions, so that they are as easy to use as possible. There are, however, a couple of compromises we had to make due…


May 19, 2016 – C++ bindings for libpmemobj (part 5) - make_persistent
One of the most important features of the C++ bindings to libpmemobj is the persistent_ptr smart pointer template. While using it is fairly straightforward, the allocation and object construction with the use of the C API is hard to get right. So like it’s C++ standard’s counterparts, it needed an…


May 10, 2016 – C++ bindings for libpmemobj (part 4) - pool handle wrapper
One of the necessary steps in developing the C++ libpmemobj bindings was the introduction of an abstraction of the C pool handle. We decided to do a very simple hierarchy where the pool template inherits from a generic pool_base. This was necessary to be able to have functions/methods which do…


February 25, 2016 – Persistent allocator design - fragmentation
Implementing a memory allocator is a balance between numerous properties with the two most important being time and space constraints. Making the malloc/free routines reasonably fast is a must for the implementation to be considered usable at all. The algorithm also mustn’t waste excessive amounts of memory. During development of…


February 22, 2016 – How to emulate Persistent Memory
Data allocated with NVML is put to the virtual memory address space, and concrete ranges are relying on result of mmap(2) operation performed on the user defined files. Such files can exist on any storage media, however data consistency assurance embedded within NVML requires frequent synchronisation of data that is…


January 12, 2016 – C++ bindings for libpmemobj (part 3) - persistent queue example
The best way to learn to code is usually by implementing an example. We are going to be creating a linked-list based queue data structure using the the nvml::obj::p and nvml::obj::persistent_ptr classes and libpmemobj C API. But first, a little bit of CS 101 :) Linked-list queue Queue is a…


January 12, 2016 – C++ bindings for libpmemobj (part 2) - persistent smart pointer
In our C API the programmer has to deal with custom pointers represented by the PMEMoid structure. Thanks to some macro magic we made it so that those PMEMoids are somewhat usable. C++ allows us to evolve this concept. nvml::obj::persistent_ptr Almost everyone who ever touched a C++ code knows the…


January 12, 2016 – C++ bindings for libpmemobj (part 1) - pmem resident variables
One of the biggest hurdles and error prone things about our C API is that the user has to manually keep track of modifications to the persistent memory resident variables while in a transaction. A special semi-transparent template property class has been implemented to automatically add variable modifications to the…


January 12, 2016 – C++ bindings for libpmemobj (part 0)
Our goal for the C pmemobj library was to make a fully featured implementation of persistent memory programming model without modifying the compiler. It’s meant for authors of very low-level system software and language creators. It’s not particularly pretty nor easy to use. The amount of macros, as well as…


December 15, 2015 – Performance improvements
I would like to inform you about the performance improvements that have been going on in NVML and libpmemobj in particular. We have not been standing still and we are trying out a couple of ideas on how to make our libraries even faster. Some of the improvements are smaller,…


November 23, 2015 – An introduction to replication
Replication is a means for raising the reliability of your pmemobj based applications. You can basically think of it as RAID 1 within NVML. What happens is, when you write to your pool using the pmemobj_* (memcpy, persist, and so on) primitives, it gets copied to your replicas. Yes, you…


October 20, 2015 – Evaluation of a better object container
During performance evaluation of our library, I asked myself a following question: Which data structure has computational complexity of “insert at end” and “remove given element” operations no worse than a doubly-linked list, but with a smaller constant? The point of that mental exercise was to come up with a…


September 29, 2015 – pmemobjfs - The simple FUSE based on libpmemobj
How to use it The sources of the pmemobjfs file system are available here. Please refer to README file for instructions on how to create a file system layout and mount it. NOTE: This is just an example implementation of file system in user space using the libpmemobj library and…


September 16, 2015 – Challenges of multi-threaded transactions
Our library currently does not support threads cooperating (writing) within a single transaction. It does shift a lot of work from the library onto the user who now has to think about different parallelization solutions. This was a conscious decision with iterative approach to creating the library in mind. It…


September 10, 2015 – KV-store improved & measured
As promised in the previous post about the kv-store implementation I’m back with new results after implementing the optimizations I devised a month ago. As a bonus I implemented a red-black tree to have a fair comparison between two data structures that allocate similar number of nodes. tl;dr: I was…


July 31, 2015 – Transactional key-value store using libpmemobj - DIY
Our library often gets compared to NoSQL databases because it stores things on storage in unstructured manner. Which is true, but, when you think about it, the pmemobj library is not technically a database, but can be used to implement one - like the MySQL storage engine example. In this…


July 20, 2015 – An introduction to pmemcheck (part 2) - transactions
In my previous blog post I described the key features of the new persistent memory analysis tool we created - pmemcheck. You should now be aware of the main pitfalls of persistent memory programming and of ways pmemcheck informs you about possible misuses of PMEM. We should now dive into…


July 17, 2015 – An introduction to pmemcheck (part 1) - basics
As you probably noticed from the previous posts, persistent memory programming isn’t really that easy. There are a couple of things you have to consider - data consistency being the most important one. The contemporary x86_64 architecture supports at most 8-byte atomic stores. You probably know by now, that by…


June 19, 2015 – An introduction to pmemobj (part 7) - persistent lists
The pmemobj library provides non-transactional persistent atomic circular doubly-linked lists (or NTPACDLL for short) API with an interface familiar to anyone who have ever included sys/queue.h header file - it’s in fact so similar that I considered not writing this post at all, you can just search the web for…


June 18, 2015 – An introduction to pmemobj (part 6) - threading
All of the pmemobj library functions are thread-safe, with following two exceptions: pool management functions (open, close and friends) and pmemobj_root when providing different sizes in different threads - so as long as you are using this function the way it’s meant to be used you don’t have to worry…


June 18, 2015 – An introduction to pmemobj (part 5) - atomic dynamic memory allocation
In the previous post I talked about using transactions for allocating new objects, which is fine and is the most similar approach to the standard POSIX way. But it does add an overhead of maintaining an undo log of changes. A more optimal memory management can be achieved using the…


June 17, 2015 – An introduction to pmemobj (part 4) - transactional dynamic memory allocation
This is a topic I intentionally avoided not to introduce too much complexity too fast. The pmemobj library contains an implemented from scratch memory allocator, that was designed with persistent memory in mind. There are two separate APIs: non-transactional and transactional. Transactional allocations Let’s start with a simple snippet of…


June 16, 2015 – An introduction to pmemobj (part 3) - types
In all of the previous post the code snippets and examples had persistent pointers (PMEMoid) without any type information - they were simple C structures. Very early in the development of the library we discovered that using something like that was extremely error-prone and generally difficult. That’s why considerable effort…


June 15, 2015 – An introduction to pmemobj (part 2) - transactions
By now you should be fairly familiar with the basics persistent memory programming, but to make sure the application is always in a consistent state you had to rely on your own solutions and tricks - like the length of a buffer in the previous example. Now, we will learn…


June 13, 2015 – An introduction to pmemobj (part 1) - accessing the persistent memory
In the previous post, you learned a little bit about the general concept of the persistent memory programming model, now it’s time to start the coding ;) Memory pools If you’ve read the NVML overview you know that persistent memory is exposed by the OS as memory-mapped files, we call…


June 12, 2015 – An introduction to pmemobj (part 0) - new programming model
The aim of this tutorial series is to introduce you to programming with persistent, byte-addressable memory using the pmemobj library. We will go over all the available features, implement an example application and learn something about the inner workings of libpmemobj. If you haven’t read the NVML overview I encourage…


June 11, 2015 – Type safety macros in libpmemobj
The PMEMoid plays the role of a persistent pointer in a pmemobj pool. It consist of a shortened UUID of the pool which the object comes from and an offset relative to the beginning of the pool: 1 2 3 4typedef struct pmemoid { uint64_t pool_uuid_lo; uint64_t off; } PMEMoid;…


June 02, 2015 – Implementing (simple) MySQL storage engine with libpmemobj
The focus of the pmemobj library, like the name suggests, is storing objects on a persistent medium. A different, but very common, approach of doing exactly the same is to use a database with a specialized interface to manipulate the collection of data. MySQL is one such database, it processes…


September 23, 2014 – Using the Block Translation Table for sector atomicity
Persistent memory based storage is able to perform IO at byte (or more accurately, cache line) granularity. However, we often want to expose such storage as traditional block devices. The block drivers for persistent memory will do exactly this. However, they do not provide any atomicity guarantees. Traditional SSDs typically…


September 09, 2014 – Git Workflow
Now that we’ve created the GitHub Repository for the NVM Library, here’s a more detailed description of the git workflow we’ve chosen. The basic idea is: Current development happens on the master branch Releases are created periodically by tagging Release branches are usually not used (master contains everything) unless it…


September 01, 2014 – NVM Library Overview
Why are we building an NVM Library? Where does it live? How does it work? This blog entry provides some answers, which refer to this picture showing the overall library architecture: Why? The operating system exposes persistent memory to applications as a memory-mapped file, using a persistent memory aware file…


August 29, 2014 – Linux Examples
The basic architecture for exposing persistent memory gives applications a very raw type of access. Applications can load/store directly to the persistence, but then what. What are the interesting problems facing an application developer and what would some solutions look like? To help describe the issues and potential solutions, we’ve…


August 27, 2014 – Crawl, Walk, Run…
If you can’t fly then run, if you can’t run then walk, if you can’t walk then crawl, but whatever you do you have to keep moving forward. Martin Luther King Jr. This project, as well as the support for persistent memory in various operating systems, can be thought of…


August 26, 2014 – References
Some very interesting research has been happening in the area of Persistent Memory and more is emerging. While the examples provided here are meant as introductory and simple, some publications cover the topic in much more depth and include complete transaction systems, compiler/language enhancements, etc. Here are some of the…


August 25, 2014 – Creating pmem.io
The pmem project in GitHub has been created as an open source project focused on persistent memory programming. Everything on this web site and the associated GitHub repositories is open source under the “three-clause” BSD license. Some educational Linux examples are included, which demonstrate the SNIA NVM programming model and…