4/25/2018 v6.82 jaf@meyersound.com
Jeremy Friesner / Meyer Sound Laboratories Inc.
Win32 compatibility contributions by Vitaliy Mikitchenko
C# client code by Wilson Yeung
SSL support code contributed by Nathan Whitehorn
Note: To compile the server, cd to the "server" subdirectory and type 'make'. The server executable, "muscled", should be generated. You can run this server from the command line, (type "muscled help" to get info about some options) and connect to it with any MUSCLE-aware program.
Alternatively, if you prefer a server that runs as a GUI program and you have Qt installed, you can cd to the "qtsupport/qt_muscled" subdirectory and type "qmake; make". A GUI server program named qt_muscled will be created.
The main goal of these classes is to provide an easy way to use BMessage-style message passing across a network of heterogeneous (MacOS/X, Linux, Windows, BSD, Unix, etc) systems. The secondary goal is just to provide some handy data containers and other utility classes.
All of this code (except the atheossupport, besupport, and qtsupport directories) should compile under any up-to-date C++ compiler-- no proprietary APIs are used except inside of the appropriate #ifdefs.
For better documentation than this, please see the MUSCLE web site.
This code has been compiled (typically with gmake, or MSVC in Windows) and tested in the following environments:
It has no known bugs, but may have some yet-to-be-discovered ones. Use at your own risk, Meyer Sound is not responsible for any disasters, blah blah blah.
Directory contents descriptions follow:
This directory contains Wilson Yeung's alpha port of the Muscle client API to the C# language. Email Wilson (wilson@whack.org) with questions about this port.
This directory contains the DataIO interface class, which is an abstraction for any file-like device that can read or write sequences of bytes, and (optionally) seek to a specified position. The DataIO class defines an interface for devices that you can Read() bytes from or Write() bytes to. This folder also includes some useful implementations of the DataIO interface, including the following:
| AsyncDataIO | Wrapper that delegates I/O calls to an internal I/O thread |
| ByteBufferDataIO | For file-style reading/writing to a ByteBuffer held in memory |
| ChildProcessDataIO | For launching a child process and communicating with it via its stdout/stdin |
| FailoverDataIO. | Wrapper for automatic fallback to a second output when a primary output fails |
| FileDataIO | For reading/writing via a C (FILE *) file handle |
| FileDescriptorDataIO | For reading/writing via a Unix file descriptor |
| MultiDataIO | Multiplexer wrapper class for writing to multiple outputs at once |
| NullDataIO | Dummy class for directing data to the bit-bucket |
| PacketizedDataIO | Wrapper for making TCP act more tr UDP |
| RS232DataIO | For communicating via an RS-232 serial port |
| SSLSocketDataIO | For communicating over SSL over TCP |
| StdinDataIO | For reading from stdin |
| TCPSocketDataIO | For communicating using a TCP socket |
| UDPSocketDataIO | For communicating using a UDP socket |
| XorDataIO | Wrapper class that applies XOR-"encryption" to all data that flows through it |
This directory contains the MUSCLE client API written for the Delphi programming environment. The contents of this directory were contributed by Matthew Emson; see the Readme.txt file in this directory for more information.
This directory contains various HTML documentation for the MUSCLE project, including the Beginner's Guide to MUSCLE, a document on how to create custom MUSCLE servers, and the autodoc folder that contains files useful for creating API documentation using the DOxygen documentation tool.
This directory contains the AbstractMessageIOGateway interface. An AbstractMessageIOGatweay is a "gateway" object that knows how to manage bidirectional FIFO Message-stream traffic going to and coming from a DataIO object. A gateway object queues outgoing MessageRefs, and when there is room in the outgoing buffer to send one, it flattens the next MessageRef in the outgoing-message-queue into a sequence of bytes, and sends those bytes out via the DataIO object. It also reads bytes coming in from the DataIO object and assembles those bytes back into Message objects, which are then handed back to the AbstractGatewayMessageReceiver specified by the calling code. This directory also contains some useful implementations of the AbstractMessageIOGateway interfaces, which are as follows:
| MessageIOGateway | Flattens Messages to the standard MUSCLE flattened-message binary format |
| PacketTunnelIOGateway | Flattens Messages into a series of fixed-size packets suitable for UDP transmission |
| PlainTextMessageIOGateway | Converts free-form lines of ASCII text into Messages, and vice versa |
| RawDataMessageIOGateway | Converts arbitrary raw data into Messages, and vice versa |
| SLIPFramedDataMessageIOGateway | Similar to the RawDataMessageIOGateway class, except it uses SLIP framing conventions |
| SignalMessageIOGateway | Dummy gateway that doesn't send actual data, only indicates when data is available |
This directory contains a Java implementation of the MUSCLE client side API. You can use the code in this folder to enable your Java program to talk to a MUSCLE server or any other program that speaks the MUSCLE Message protocol.
This directory contains MUSCLE's Message class. A Message is a general-purpose data structure that is similar to BeOS's BMessage class. A Message consists of a 32-bit integer "what code", plus zero or more named data fields, each of which can contain one or more data items of a specified type.
Here are some relevant details:
| int8 | B_INT8_TYPE | 8-bit signed integer values | |
| int16 | B_INT16_TYPE | 16-bit signed integer values | |
| int32 | B_INT32_TYPE | 32-bit signed integer values | |
| int64 | B_INT64_TYPE | 64-bit signed integer values | |
| bool | B_BOOL_TYPE | boolean values | |
| float | B_FLOAT_TYPE | IEEE 754 floating point values | |
| Pointer | B_POINTER_TYPE | Pointer values (non-flattenable) | |
| Message | B_MESSAGE_TYPE | Message objects | |
| Flattenable | (various types) | Flattened Flattenable objects | |
| String | B_STRING_TYPE | UTF8 character strings | |
| Rect | B_RECT_TYPE | Rectangles (floats for left,top,right,bottom) | |
| Point | B_POINT_TYPE | Points (floats for x,y) | |
| Raw Data Buffer | B_RAW_TYPE | Sequences of zero or more untyped bytes | |
| Tag | B_TAG_TYPE | User-provided arbitrary objects (non-flattenable) |
This directory contains the MiniMessage and MiniMessageGateway C APIs. These APIs are C implementations of the C++ Message and MessageIOGateway classes. They can be used in cases where (for whatever reason) you want to code your program in C only and avoid C++. They aren't as easy-to-use as the C++ implementation, but they should be sufficient for simple things, and they compile down to only a few dozen kilobytes of object code. See the testmini.c and minireflectclient.c test files in the tests directory for examples on how they are used.
This directory contains the MicroMessage and MicroMessageGateway C APIs. These APIs are C implementations of the C++ Message and MessageIOGateway classes. These APIs go even farther towards minimalism than the minimessage APIs: in particular, these APIs never allocate or free any data. Instead of converting the message's flattened-data-bytes into a separate in-memory data structure like Message and MiniMessage do, MicroMessage operates on the flattened-data-bytes directly. This makes for a potentially much more efficient implementation; the main downside is that when creating a MicroMessage, you can only append data; you cannot insert or remove fields that you previously added.
This directory contains a minimal MUSCLE client-side API written in Python. You can use the code in this directory to enable your Python scripts to talk to a MUSCLE server or any other program that speaks the MUSCLE Message protocol. Also included in this directory is some C++ glue code (in PythonUtilityFunctions.cpp) that is useful when embedding Python code into C++ code -- the glue code uses MUSCLE Messages as to transfer arguments from C++ to Python context and back again.
This directory contains several classes that support clients that use TrollTech's Qt cross-platform GUI API. The main one is the QMessageTransceiverThread class, which is a Qt-aware subclass of the MessageTransceiverThread class. Using a QMessageTransceiverThread for your network I/O makes network communication very simple; instead of dealing with bytes and network protocols, you simply receive a Qt signal whenever incoming Messages are available, and call a method to send a Message, etc. This folder also contains some sub-directories that contain small example programs written for MUSCLE+Qt:
| qt_example | A simple multi-user 'game' and chat program |
| qt_advanced_example | A demonstration of embedding a MUSCLE server thread inside a Qt application |
| qt_muscled | A demonstration of a Qt app that runs a MUSCLE server as a child process |
| qt_muscled_browser | A hierarchical browser GUI for seeing what data is present in a muscle server's database |
This directory contains server code for an n-way "message crossbar server" program. This program will listen on a specified port for TCP connections, and will allow the TCP connections to "talk to each other" by forwarding Messages from one client to another (or to multiple others). The ServerProcessLoop() method implements the server's event loop, while the AbstractReflectSession class is the interface for the server's side of a TCP connection. There are currently two subclasses of AbstractReflectSession included: the DumbReflectSession class just reflects all received Messages to all connected clients, while the StorageReflectSession class adds nice features like wildcard-based Message routing, server-side data storage, and "notify-me-on-change" subscription services. (See the MUSCLE Beginner's Guide for more info on this) More elaborate logic can be added by creating subclasses of these classes.
This directory contains code to support the use of regular expressions. This includes some C++ pattern-matching utility classes, as well as a sub-folder containing Henry Spencer's freeware C regex engine, for use in OS's that do not provide their own regex library.
Classes implemented in this directory include:
| FilePathExpander | Expands shell-style wildcards into a list of file paths |
| PathMatcher | Controls wildcard-directed recursive iterations down the tree of DataNode objects |
| QueryFilter | Implements various predicate-logic operations on DataNode Message data |
| StringMatcher | Does shell-style pattern matching on arbitrary character strings |
| SegmentedStringMatcher | Like StringMatcher, except that the strings are divided up into substrings which are evaluated separately (e.g. "a*/b*") |
This directory contains the SDLMessageTransceiverThread class, which is a handy way to implement MUSCLE communication ability into your SDL program. SDLMessageTransceiverThread class is just a thin wrapper subclass around the MessageTransceiverThread class, but it interfaces MessageTransceiverThread to SDL's event-notification system.
This contains the Makefile and main entry point for the "muscled" server executable, and the "admin" muscled-server-administration utility.
This directory contains various "small things" needed to compile the rest of the code. These include byte-ordering macros, BeOS-style type codes, typedefs, and result constants, and the Flattenable interface definition.
This directory contains functions for logging event messages to stdout and/or to a file. Log messages can be "raw" (works just like printf) or nicely formatted with the current time, redirected to a file, and so on. The logging system also has optional functionality to rotate, compress and/or delete old log files, to avoid filling up too much disk space.
This directory contains classes that represent "generic" interfaces to OS-specific APIs; as such, they are not guaranteed to work on every platform. Currently this directory contains the following classes:
| AcceptSocketsThread | A thread that accepts incoming TCP connections and hands them back to the parent thread |
| AtomicCounter | Atomic-increment and atomic-decrement counter, for lock-free reference-counting |
| DetectNetworkConfigChangesSession | A session object that notifies the program when the host computer's network configuration has changed |
| GlobalMemoryAllocator | Code to monitor and optionally restrict the program's heap usage |
| MessageTransceiverThread | Runs a MUSCLE ReflectServer object in a separate thread. Provides asynchronous I/O |
| Mutex | Provides efficient in-process locking (aka critical sections) for thread-safety |
| SetupSystem | Provides standardized startup and shutdown routines that must be used by any MUSCLE process |
| SharedMemory | Implements inter-process shared memory regions, including inter-process read/write locking ability |
| SignalMultiplexer | Makes system signalling (e.g. catching of SIGINT or SIHUP) available to multiple pieces of code in the same process |
| SystemInfo | Provides information about the environment the program is operating in (current directory, OS version, etc) |
| Thread | An OS-neutral Thread class for multithreading purposes. Includes send/receive Message functionality for easy control |
| ThreadLocalStorage | An OS-neutral implementation of thread-local data storage |
| ThreadPool | A thread pool implementation to allow handling of many Messages in parallel across a finite number of threads. |
This directory contains various test programs that I use to test and develop the code, and a Makefile to build them with.
Currently this directory contains the following programs:
| bandwidthtester | Generates lots of Message traffic and measures how fast a MUSCLE server can receive it |
| calctypecode | Prints out the decimal equivalent of a four-character ASCII what-code |
| printtypecode | Prints out the four-character ASCII equivalent of a given decimal value |
| chatclient | Minimalist BeShare-compatible chat client that can connect to any MUSCLE server |
| deadlock | Deliberately tries to create a deadlock. Primarily used for testing deadlockfinder. |
| deadlockfinder | Parses the output generated by MUSCLE's MUSCLE_ENABLE_DEADLOCK_FINDER feature, and detects potential deadlock issues |
| findsourcelocations | Parses source code and lists source-locations matching a specified code generated by MUSCLE's MUSCLE_INCLUDE_SOURCE_CODE_LOCATION_IN_LOGTIME feature |
| hexterm | A simple interactive terminal that sends, receives, and prints hexadecimal representation of all bytes received via TCP, UDP, etc. |
| portableplaintextclient | A simple interactive terminal for ASCII communication over TCP |
| portablereflectclient | A simple interactive terminal for Message communication over TCP (typically with a MUSCLE server) |
| portscan | Attempts to connect via TCP to a range of ports on a host, and reports which ports accepted the connection |
| printsourcelocations | Prints the source-code-location codes of all LogTime() commands. Useful for building up a directory of source-location-codes for a given codebase |
| readmessage | Reads a flattened Message from a file on disk and prints it to stdout in human-readable form |
| serialproxy | Makes a local serial port available to the network over TCP |
| svncopy | Creates a command script for bulk-adding specified files to an SVN repository |
| testatheossupport | Tests the AtheOS support files in the atheossupport folder |
| testbatchguard | Unit test for the BatchGuard class |
| testbesupport | Tests the BeOS support files in the beossupport folder |
| testbytebuffer | Unit test for the ByteBuffer class |
| testchildprocess | Unit test for the ChildProcessDataIO class |
| testendian | Unit test for the endian-swapping routines |
| testfilepathinfo | Unit test for the FilePathInfo class |
| testgateway | Unit test for the MessageIOGateway class |
| testhashtable | Unit test for the Hashtable class |
| testmatchfiles | Unit test for the ExpandFilePathWildCards() function |
| testmessage | Unit test for the Message class |
| testmicro | Unit test for the MicroMessage and MicroMessageGateway C routines |
| testmini | Unit test for the MiniMessage and MiniMessageGateway C routines |
| testnagle | Unit test to verify the presence of absence of Nagle's algorithm |
| testnetconfigdetect | Unit test for the DetectNetworkConfigChangesSession class |
| testnetutil | Unit test for the GetNetworkInterfaces() function |
| testpacketio | Unit test for the PacketizedDataIO clas |
| testpackettunnel | Unit test for the PacketTunnelIOGateway class |
| testparsefile | Unit test for the ParseFile() and ParseArgs() functions |
| testpool | Unit test for the ObjectPool class |
| testpulsenode | Unit test for the PulseNode class |
| testqueryfilter | Unit test for the QueryFilter classes |
| testqueue | Unit test for the Queue class |
| testrefcount | Unit test for the RefCount class |
| testreflectclient | Unit test for the various OS-specific MessageTransceiverThread subclasses |
| testregex | Unit test for the StringMatcher class |
| testresponse | Test to measure the response latency of a muscle server |
| testserial | Unit test for the RS232DataIO class |
| testsharedmem | Unit test for the SharedMemory class |
| testsocketmultiplexer | Unit test for the SocketMultiplexer class |
| teststring | Unit test for the String class |
| testsysteminfo | Unit test for the SystemInfo functionality |
| testthread | Unit test for the Thread class |
| testthreadpool | Unit test for the ThreadPool class |
| testtime | Unit test for the various time-string-parsing and time-string-generation functions |
| testtuple | Unit test for the Tuple class |
| testtypedefs | Unit test for MUSCLE's standard type typedefs (int32, int64, etc) |
| testudp | Test/demonstration of using MUSCLE to send/receive UDP packets |
| testzip | Unit test of the ReadZipFile() and WriteZipFile() functions |
| udpproxy | Forwards UDP packets from one computer to another, and vice versa |
| uploadstress | Spams a MUSCLE server with requests to see if the server can keep up |
| win32client | Example of integrating MUSCLE client code with a Win32 event loop |
This directory contains many useful one-off classes and function collections, including:
BatchOperator is a templated mechanism for ensuring that a specified method gets called on the first-level recursion into a call tree, and that a matching method gets called on the final recursion out of the call tree. This is handy for making sure that setup and shutdown code is called a the correct times.
ByteBufferByteBuffer is an intelligent byte-array class, that stores its length, knows how to resize itself efficiently, is reference-countable and flattenable, etc.
CountedObject is a class that other classes can be derived from if you want to keep track of how many instances of them are in memory at any given time.
CPULoadMeter reports the percentage of CPU time being used on the local computer from moment to moment (similar to what is reported in Task Manager under Windows, or Activity Monitor under MacOS/X)
Directory is a platform-neutral API for scanning a filesystem directory and iterating over its contents.
DebugTimer is a useful utility class that is useful for debugging performance problems. It records the current time in its constructor, and then in its destructor it prints out the time that has elapsed since then (if the elapsed time is more than a specified minimum time).
FilePathInfo is a platform-neutral API for querying the various properties of an entry at a specified location in a file system (e.g. is a file, a directory, or a symlink, how large is it, when was it created or modified, etc)
Hashtable is a handy hash table class, with templated key and value types and the traditional O(1) lookup time. In addition to that, it includes other nice features, such as "safe" iterators (so you can modify the Hashtable while iterating through it), minimal-frequency memory allocations, and the ability to sort the table by key or by value (it maintains the ordering of the objects placed into the table). Hashtable is used by the Message class, but is also quite useful on its own.
IPAddress is a class that represents an IPv4 or IPv6 address in a standardized manner. There is also an IPAddressAndPort class which represents the combination of an IP address and a port number.
NestCount is a simple class for recording when the execution path enters or exits a particular function, and (optionally) making decisions based on whether a specified function is currently on the stack or not.
NetworkUtilityFunctions.cpp is a repository of user-friendly wrapper functions around common BSD socket operations, such as setting up sockets to connect or accept TCP connections, or binding a UDP socket to a port.
The ObjectPool class is used to avoid excessive deletions and allocations of commonly used objects (such as Messages or RefCountMems). It works by recycling the items for re-use, and is templated so it can be used for any type of object.
The PulseNode is an interface for objects that want to execute a particular action at a specified time. It works in conjunction with the standard MUSCLE event loop. Implementing classes define a Pulse() method that will be executed at a specified time, and a GetPulseTime() method that returns a clock value indicating when Pulse() should next be executed.
The Queue class is just a little templatized double-ended queue (i.e. AddHead(), AddTail(), RemoveHead(), and RemoveTail() are O(1) operations). It can be used as a Vector, Stack, or FIFO. It's templatized for easy, type-safe reuse.
The RefCount class implements generic reference counting for C++ objects or arrays. To enable reference counting on an object, you simply create a single Ref for that object, and (optionally) make one or more copies of the Ref via the copy constructor or the equals operator. Then, when the last Ref object disappears, the C++ object or array is automatically deleted. It's not a garbage collector, but it beats having to keep track of all your allocations by hand...
A reference-countable C++ class wrapper for a socket or file descriptor. Wrapping sockets and file descriptors in these objects allows them to be easily shared across objects without introducing the possibility of leaking them, or closing them before some other piece of code is done using them.
An easy-to-use wrapper around the select() socket-multiplexing API. It's also possible to use this as a wrapper around various select() alternatives, such as poll(), epoll(), or kqueue(), by recompiling the code with the appropriate compiler flags (e.g. -DMUSCLE_USE_POLL)
The String class is just your basic character-string class, in this case inspired by the java.lang.String class from Java. This class was originally written by Michael Olivero (mike95@mike95.com) and modified by myself. String extends Flattenable, so it can be serialized in a generic manner.
A string tokenizing class similar to Java's Java.util.StringTokenizer, only more efficient.
TimeUtilityFunctions.h is a repository of functions for dealing with microsecond-accurate timing issues.
This directory contains project files for building muscled under Visual C++ for Windows. These files were provided by Vitaliy Mikitchenko (aka VitViper)
This directory contains the Win32MessageTransceiverThread class, which is useful for interface MUSCLE code to the standard Win32 GUI event loop. You can use this class to enable your Win32 C and C++ programs to communicate with a MUSCLE server or any other program that speaks the MUSCLE Message protocol.
This directory contains a subfolder named zlib (which contains the complete source code of the zlib compressor/decompressor package). This directory also contains some zlib-related muscle source, including ZLibCodec, which is a convenience class for compressing and decompressing chunks of data, ZLibDataIO, a wrapper class for transparent compression and decompression of I/O streams, and ZLibUtilityFunctions, which contain some convenience functions for quickly compressing and decompressing a Message in a compatible manner.
For more details, have a look at the autodocs, header files and/or the source itself.
-Jeremy