1// Copyright (C) 2022 Intel Corporation.
2// SPDX-License-Identifier: LicenseRef-Qt-Commercial OR GFDL-1.3-no-invariants-only
6 \title Inter-Process Communication
8 \ingroup frameworks-technologies
11 \brief An overview of Qt's inter-process communication functionality
13 Qt supports many ways of communicating with other processes running in the
14 same system or in different systems. There are basically three types of
15 inter-process communication mechanisms:
18 \li Synchronization primitives
19 \li Exchanging of arbitrary byte-level data
20 \li Passing structured messages
23 \section1 Synchronization primitives
25 Qt only provides one class for explicit inter-process synchronization:
26 \l{QSystemSemaphore}. A QSystemSemaphore is like a \l{QSemaphore} that is
27 accessible by multiple processes in the same system. It is globally
28 identified by a "key", which in Qt is represented by the \l{QNativeIpcKey}
29 class. Additionally, depending on the OS, Qt may support multiple different
30 backends for sharing memory; see the \l{Native IPC Keys} documentation for
31 more information and limitations.
33 It is possible to use regular thread-synchronization primitives such as
34 mutexes, wait conditions, and read-write locks, located in memory that is
35 shared between processes. Qt does not provide any class to support this,
36 but applications can use low-level operations on certain operating systems.
38 Other Qt classes may be used to provide higher-level locking, like
39 \l{QLockFile}, or by acquiring a unique, system-wide resource. Such
40 techniques include \l{QTcpServer}{TCP} or \l{QUdpSocket}{UDP} ports or
41 well-known names in \l{QDBusConnection::registerService}{D-Bus}.
43 \section1 Byte-level data sharing
45 Using byte-level data, applications can implement any communication
46 protocol they may choose. Sharing of byte data can be stream-oriented
47 (serialized) or can allow random access (a similar condition to
48 QFileDevice::isSequential()).
50 For serial communication, Qt provides a number of different classes and
53 \li Pipes and FIFOs: \l QFile
54 \li Child processes: \l QProcess
55 \li Sockets: \l QTcpSocket, \l QUdpSocket (in \l{Qt Network})
56 \li HTTP(S): \l QNetworkAccessManager (in \l{Qt Network}) and
57 \l QHttpServer (in \l{Qt HTTP Server})
58 \li CoAP(S): \l QCoapClient (in \l{Qt CoAP})
61 For random-access data sharing within the same system, Qt provides
62 \l{QSharedMemory}. See the \l{Shared Memory} documentation for detailed
65 \section1 Structured message passing
67 Qt also provides a number of techniques to exchange structured messages
68 with other processes. Applications can build on top of the byte-level
69 solutions above, such as by using \l QJsonDocument or \l QXmlStreamReader /
70 \l QXmlStreamWriter over HTTP to perform JSONRPC or XMLRPC, respectively,
71 or \l QCborValue with QtCoAP.
73 Dedicated Qt modules for structured messages and remote procedure-calling
77 \li \l{Qt Remote Objects}
83 \page shared-memory.html
86 \keyword shared memory
88 \brief Overview of the techniques for sharing memory between processes
90 Qt provides two techniques to share memory with other processes in the same
91 system: \l{QSharedMemory} and memory-mapped files using \l{QFile}. Memory
92 that is shared with other processes is often referred to as a "segment",
93 and although it may have been implemented as specific segments on
94 processors with segmented memory models in the past, this is not the case
95 in any modern operating system. Shared memory segments are simply regions
96 of memory that the operating system will ensure are available to all
97 processes participating.
99 \note The address at which the segment is located in memory will almost
100 always be different for each process that is participating in the sharing.
101 Therefore, applications must take care to share only position-independent
102 data, such as primitive C++ types or arrays of such types.
104 \section1 Sharing memory using QSharedMemory
106 QSharedMemory provides a simple API to create a shared memory segment of a
107 given size or attach to one that was created by another process.
108 Additionally, it provides a pair of methods to \l{QSharedMemory::}{lock}
109 and \l{QSharedMemory::}{unlock} the whole segment, using an internal
110 \l{QSystemSemaphore}.
112 Shared memory segments and system semaphores are globally identified in the
113 system through a "key", which in Qt is represented by the \l{QNativeIpcKey}
114 class. Additionally, depending on the OS, Qt may support multiple different
115 backends for sharing memory; see the \l{Native IPC Keys} documentation for
116 more information and limitations.
118 QSharedMemory is designed to share memory only within the same privilege
119 level (that is, not with untrusted other processes, such as those started
120 by other users). For backends that support it, QSharedMemory will create
121 segments such that only processes with the same privilege level can attach.
123 \section1 Sharing memory via memory-mapped files
125 Most files can be mapped to memory using QFile::map() and, if the
126 \l{QFileDevice::MapPrivateOption}{MapPrivateOption} option is not specified,
127 any writes to the mapped segment will be observed by all other processes
128 that have mapped the same file. Exceptions to files that can be mapped to
129 memory include remote files found in network shares or those located in
130 certain filesystems. Even if the operating system does allow mapping remote
131 files to memory, I/O operations on the file will likely be cached and
132 delayed, thus making true memory sharing impossible.
134 This solution has the major advantages of being independent of any backend
135 API and of being simpler to interoperate with from non-Qt applications.
136 Since \l{QTemporaryFile} is a \l{QFile}, applications can use that class to
137 achieve clean-up semantics and to create unique shared memory segments too.
139 To achieve locking of the shared memory segment, applications will need to
140 deploy their own mechanisms. One way may be to use \l QLockFile. Another
141 and less costly solution is to use QBasicAtomicInteger or \c{std::atomic} in
142 a pre-determined offset in the segment itself. Higher-level locking
143 primitives may be available on some operating systems; for example, on
144 Linux, applications can set the "pshared" flag in the mutex attribute
145 passed to \c{pthread_mutex_create()} to indicate that the mutex resides in
146 a shared memory segment.
148 Be aware that the operating system will likely attempt to commit to
149 permanent storage any writes made to the shared memory. This may be desired
150 or it may be a performance penalty if the file itself was meant to be
151 temporary. In that case, applications should locate a RAM-backed
152 filesystem, such as \c{tmpfs} on Linux (see
153 QStorageInfo::fileSystemType()), or pass a flag to the native file-opening
154 function to inform the OS to avoid committing the contents to storage.
156 It is possible to use file-backed shared memory to communicate with
157 untrusted processes, in which case the application should exercise great
158 care. The files may be truncated/shrunk and cause applications accessing
159 memory beyond the file's size to crash.
161 \section2 Linux hints on memory-mapped files
163 On modern Linux systems, while the \c{/tmp} directory is often a \c{tmpfs}
164 mount point, that is not a requirement. However, the \c{/dev/shm} directory
165 is required to be a \c{tmpfs} and exists for the very purpose of sharing
166 memory. Do note that it is world-readable and writable (like \c{/tmp} and
167 \c{/var/tmp}), so applications must be careful of the contents revealed
168 there. Another alternative is to use the XDG Runtime Directory (see
169 QStandardPaths::writableLocation() and \l{QStandardPaths::RuntimeLocation}),
170 which on Linux systems using systemd is a user-specific \c{tmpfs}.
172 An even more secure solution is to create a "memfd" using \c{memfd_create(2)}
173 and use interprocess communication to pass the file descriptor, like
174 \l{QDBusUnixFileDescriptor} or by letting the child process of a \l{QProcess}
175 inherit it. "memfds" can also be sealed against being shrunk, so they are
176 safe to be used when communicating with processes with a different privilege
179 \section2 FreeBSD hints on memory-mapped files
181 FreeBSD also has \c{memfd_create(2)} and can pass file descriptors to other
182 processes using the same techniques as Linux. It does not have temporary
183 filesystems mounted by default.
185 \section2 Windows hints on memory-mapped files
187 On Windows, the application can request the operating system avoid saving
188 the file's contents on permanent storage. This request is performed by
189 passing the \c{FILE_ATTRIBUTE_TEMPORARY} flag in the
190 \c{dwFlagsAndAttributes} parameter to the \c{CreateFile} Win32 function,
191 the \c{_O_SHORT_LIVED} flag to \c{_open()} low-level function, or by
192 including the modifier "T" to the
193 \c{fopen()} C runtime function.
195 There's also a flag to inform the operating system to delete the file when
196 the last handle to it is closed (\c{FILE_FLAG_DELETE_ON_CLOSE},
197 \c{_O_TEMPORARY}, and the "D" modifier), but do note that all processes
198 attempting to open the file must agree on using this flag or not using it. A
199 mismatch will likely cause a sharing violation and failure to open the file.
203 \page native-ipc-keys.html
204 \title Native IPC Keys
206 \keyword shared memory
207 \keyword system semaphore
209 \brief An overview of keys for QSharedMemory and QSystemSemaphore
211 The \l QSharedMemory and \l QSystemSemaphore classes identify their
212 resource using a system-wide identifier known as a "key". The low-level key
213 value as well as the key type are encapsulated in Qt using the \l
214 QNativeIpcKey class. That class also provides the proper means of
215 exchanging the key with other processes, by way of
216 QNativeIpcKey::toString() and QNativeIpcKey::fromString().
218 Qt currently supports three distinct backends for those two classes, which
219 match the values available in the \l{QNativeIpcKey::Type} enumeration.
221 \li POSIX Realtime extensions (IEEE 1003.1b, POSIX.1b)
222 \li X/Open System Interfaces (XSI) or System V (SVr4), though also now part of POSIX
223 \li Windows primitives
226 As the name indicates, the Windows primitives are only available on the
227 Windows operating system, where they are the default backend. The other two
228 are usually both available on Unix operating systems. The following table
229 provides an overview of typical availability since Qt 6.6:
232 \header \li Operating system \li POSIX \li System V \li Windows
233 \row \li Android \li \li \li
234 \row \li INTEGRITY \li \li \li
235 \row \li QNX \li Yes \li \li
236 \row \li \macos \li Yes \li Usually (1) \li
237 \row \li Other Apple OSes \li Yes \li \li
238 \row \li Other Unix systems \li Yes \li Yes \li
239 \row \li Windows \li Rarely (2) \li \li Yes
242 \note 1 Sandboxed \macos applications, which include all applications
243 distributed via the Apple App Store, may not use System V objects.
245 \note 2 Some GCC-compatible C runtimes on Windows provide POSIX-compatible
246 shared memory support, but this is rare. It is always absent with the
249 To determine whether a given key type is supported, applications should
250 call QSharedMemory::isKeyTypeSupported() and
251 QSystemSemaphore::isKeyTypeSupported().
253 QNativeIpcKey also provides support for compatibility with Qt applications
254 prior to its introduction. The following sections detail the limitations of
255 the backends, the contents of the string keys themselves, and
258 \section1 Cross-platform safe key format
260 QNativeIpcKey::setNativeKey() and QNativeIpcKey::nativeKey() handle the
261 low-level native key, which may be used with the native APIs and shared
262 with other, non-Qt processes (see below for the API). This format is not
263 usually cross-platform, so both QSharedMemory and QSystemSemaphore provide
264 a function to translate a cross-platform identifier string to the native
265 key: QSharedMemory::platformSafeKey() and
266 QSystemSemaphore::platformSafeKey().
268 The length of the cross-platform key on most platforms is the same as that
269 of a file name, but is severely limited on Apple platforms to only 30
270 usable bytes (be mindful of UTF-8 encoding if using characters outside the
271 US-ASCII range). The format of the key is also similar to that of a file
272 path component, meaning it should not contain any characters not allowed in
273 file names, in particular those that separate path components (slash and
274 backslash), with the exception of sandboxed applications on Apple operating
275 systems. The following are good examples of cross-platform keys: "myapp",
276 "org.example.myapp", "org.example.myapp-12345".
278 \b{Apple sandbox limitations:} if the application is running inside of a
279 sandbox in an Apple operating system, the key must be in a very specific
280 format: \c {<application group identifier>/<custom identifier>}. Sandboxing
281 is implied for all applications distributed through the Apple App Store.
282 See Apple's documentation
283 \l{https://developer.apple.com/library/archive/documentation/Security/Conceptual/AppSandboxDesignGuide/AppSandboxInDepth/AppSandboxInDepth.html#//apple_ref/doc/uid/TP40011183-CH3-SW24}
284 {here} and \l{https://developer.apple.com/documentation/bundleresources/entitlements/com_apple_security_application-groups}
285 {here} for more information, including how to obtain the application's group identifier.
287 \section1 Native key format
289 This section details the format of the native keys of the supported
292 \section3 POSIX Realtime
293 Native keys resemble file names and may contain any character that file
294 names do, except for a slash. POSIX requires the first character in the key
295 name to be a slash and leaves undetermined whether any additional slashes
296 are permitted. On most operating systems, the key length is the same as a
297 file name, but it is limited to 32 characters on Apple operating systems
298 (this includes the first slash and the terminating null, so only 30 usable
299 characters are possible).
301 The following are good examples of native POSIX keys: "/myapp",
302 "/org.example.myapp", "/org.example.myapp-12345".
304 QSharedMemory::platformSafeKey() and QSystemSemaphore::platformSafeKey()
305 simply prepend the slash. On Apple operating systems, they also truncate
306 the result to the available size.
309 Windows key types are NT
310 \l{https://learn.microsoft.com/en-us/windows/win32/sync/object-namespaces}{kernel
311 object names} and may be up to \c{MAX_PATH} (260) characters in length.
312 They look like relative paths (that is, they don't start with a backslash
313 or a drive letter), but unlike file names on Windows, they are
316 The following are good examples of native Windows keys: "myapp",
317 "org.example.myapp", "org.example.myapp-12345".
319 QSharedMemory::platformSafeKey() and QSystemSemaphore::platformSafeKey()
320 insert a prefix to disambiguate shared memory and system semaphores,
323 \section3 X/Open System Interfaces (XSI) / System V
324 System V keys take the form of the name of a file in the system, and thus
325 have the exact same limitations as file paths do. Both QSharedMemory and
326 QSystemSemaphore will create this file if it does not exist when creating
327 the object. If auto-removal is disabled, it may also be shared between
328 QSharedMemory and QSystemSemaphore without conflict and can be any extant
329 file (for example, it can be the process executable itself, see
330 QCoreApplication::applicationFilePath()). The path should be an absolute
331 one to avoid mistakes due to different current directories.
333 QSharedMemory::platformSafeKey() and QSystemSemaphore::platformSafeKey()
334 always return an absolute path. If the input was already absolute, they
335 will return their input unchanged. Otherwise, they will prepend a suitable
336 path where the application usually has permission to create files in.
340 Shared memory and system semaphore objects need to be created before use,
341 which is accomplished with QSharedMemory::create() or by passing
342 QSystemSemaphore::Create to the constructor, respectively.
344 On Unix systems, the Qt classes that created the object will be responsible
345 for cleaning up the object in question. Therefore, if the application with
346 that C++ object exits uncleanly (a crash, qFatal(), etc.), the object may
347 be left behind. If that happens, applications may fail to create the
348 object again and should instead attach to an existing one. For example, for
352 if (!shm.create(4096) && shm.error() == QSharedMemory::AlreadyExists)
356 Re-attaching to a QSystemSemaphore is probably unwise, as the token counter
357 in it is probably in an unknown state and therefore may lead to deadlocks.
359 \section3 POSIX Realtime
360 POSIX Realtime object ownership is patterned after files, in the sense that
361 they exist independent of any process using them or not. Qt is unable to
362 determine if the object is still in use, so auto-removal will remove it
363 even then, which will make attaching to the same object impossible but
364 otherwise not affecting existing attachments.
366 Prior to Qt 6.6, Qt never cleaned up POSIX Realtime objects, except on QNX.
368 \section3 X/Open System Interfaces (XSI) / System V
369 There are two resources managed by the Qt classes: the file the key refers
370 to and the object itself. QSharedMemory manages the object cooperatively:
371 the last attachment is responsible for removing the object itself and then
372 removing the key file. QSystemSemaphore will remove the object if and only
373 if it was passed QSystemSemaphore::Create; additionally, if it created the
374 key file, it will remove that too.
376 Since Qt 6.6, it is possible to ask either class not to clean up.
379 The operating system owns the object and will clean up after the last
380 handle to the object is closed.
382 \section1 Interoperability with old Qt applications
384 The QNativeIpcKey class was introduced in Qt 6.6. Prior to this version,
385 QSharedMemory and QSystemSemaphore backends were determined at the time of
386 Qt's own build. For Windows systems, it was always the Windows backend. For
387 Unix systems, it defaulted to the System V backend if the configuration
388 script determined it was available. If it was not available, it fell back
389 to the POSIX one (since Qt 4.8). The POSIX backend could be explicitly
390 selected using the \c{-feature-ipc_posix} option to the Qt configure
391 script; if it was enabled, the \c{QT_POSIX_IPC} macro would be defined.
393 Qt 6.6 retains the configure script option but it no longer controls the
394 availability of the backends. Instead, it changes what
395 QNativeIpcKey::legacyDefaultTypeForOs() will return. Applications that need
396 to retain compatibility must use this key type exclusively to guarantee
399 The API in both QSharedMemory and QSystemSemaphore had the concept of a
400 cross-platform key, which is now deprecated in favor of using
401 QSharedMemory::legacyNativeKey() and QSystemSemaphore::legacyNativeKey().
402 Those two functions produce the same native key as the deprecated functions
403 did in prior versions. If the old code was for example:
406 QSharedMemory shm("org.example.myapplication");
407 QSystemSemaphore sem("org.example.myapplication");
410 It can be updated to be:
412 QSharedMemory shm(QSharedMemory::legacyNativeKey("org.example.myapplication"));
413 QSystemSemaphore sem(QSystemSemaphore::legacyNativeKey("org.example.myapplication"));
416 If the two applications exchanged native keys, there is no need to update
421 shm.setNativeKey(key);
424 Though if the older application did accept a native key, the new one may
425 opt to use \c{platformSafeKey()} with a second argument of
426 QNativeIpcKey::legacyDefaultTypeForOs().
428 \section3 X/Open System Interfaces (XSI) / System V
429 Never use existing files for QSharedMemory keys, as the old Qt application
430 may attempt to remove it. Instead, let QSharedMemory create it.
432 \section1 Interoperability with non-Qt applications
434 Interoperability with non-Qt applications is possible, with some limitations:
436 \li Creation of shared memory segments must not race
437 \li QSharedMemory support for locking the segment is unavailable
440 Communication with non-Qt applications must always be through the native
443 QSharedMemory always maps the entire segment to memory. The non-Qt
444 application may choose to only map a subset of it to memory, with no ill
447 \section3 POSIX Realtime
448 POSIX shared memory can be opened using
449 \l{https://pubs.opengroup.org/onlinepubs/9699919799/functions/shm_open.html}{shm_open()}
450 and POSIX system semaphores can be opened using
451 \l{https://pubs.opengroup.org/onlinepubs/9699919799/functions/sem_open.html}{sem_open()}.
453 Both of those functions take a \c name parameter that is the result of
454 QNativeIpcKey::nativeKey(), encoded for file names using
455 QFile::encodeName() / QFile::decodeName().
458 Windows shared memory objects can be opened using
459 \l{https://learn.microsoft.com/en-us/windows/win32/api/memoryapi/nf-memoryapi-createfilemappingw}{CreateFileMappingW}
460 and Windows system semaphore objects can be opened using
461 \l{https://learn.microsoft.com/en-us/windows/win32/api/synchapi/nf-synchapi-createsemaphorew}{CreateSemaphoreW}.
462 Despite the name of both functions starting with "Create", they are able
463 to attach to existing objects.
465 The \c lpName parameter to those functions is the result of
466 QNativeIpcKey::nativeKey(), without transformation.
468 If the foreign application uses the non-Unicode version of those functions
469 (ending in "A"), the name can be converted to and from 8-bit using QString.
471 \section3 X/Open System Interfaces (XSI) / System V
472 System V shared memory can be obtained using
473 \l{https://pubs.opengroup.org/onlinepubs/9699919799/functions/shmget.html}{shmget()}
474 and System V system semaphores can be obtained using
475 \l{https://pubs.opengroup.org/onlinepubs/9699919799/functions/semget.html}{semget()}.
477 The \c{key} parameter to either of those functions is the result of the
478 \l{https://pubs.opengroup.org/onlinepubs/9699919799/functions/ftok.html}{ftok()}
479 function when passed the file name obtained from QNativeIpcKey::nativeKey()
480 with an \c id of 81 or 0x51 (the ASCII capital letter 'Q').
482 System V semaphore objects may contain multiple semaphores, but
483 QSystemSemaphore only uses the first one (number 0 for \c{sem_num}).
485 Both QSharedMemory and QSystemSemaphore default to removing the object
486 using the \c{IPC_RMID} operation to \c{shmctl()} and \c{semctl()}
487 respectively if they are the last attachment.
1.9.7