diff options
Diffstat (limited to 'README.md')
| -rw-r--r-- | README.md | 115 | 
1 files changed, 115 insertions, 0 deletions
| diff --git a/README.md b/README.md new file mode 100644 index 0000000..f5302a8 --- /dev/null +++ b/README.md | |||
| @@ -0,0 +1,115 @@ | |||
| 1 | # Generalized Agile Transport System (GATS) | ||
| 2 | |||
| 3 | GATS is a data encoding format, as well as a set of libraries for reading and | ||
| 4 | writing data in that format. It's a structured format reminiscent of a binary | ||
| 5 | json, but with some extra cool features. | ||
| 6 | |||
| 7 | * All numbers (integers and floating point) are stored in an arbitrary | ||
| 8 | precision format which takes up the least amount of space it can. | ||
| 9 | * Unlike textual formats (like json, xml, etc.) floating point numbers are | ||
| 10 | stored exactly, the exact number you write in is the exact number you read | ||
| 11 | out. | ||
| 12 | * All formats stored are machine independent, you can read and write them on | ||
| 13 | any architecture, any endianness and they will always be the same. No more | ||
| 14 | encoding worries. | ||
| 15 | * There are a number of data types you can use to make up data structures: | ||
| 16 | * Dictionaries | ||
| 17 | * Lists | ||
| 18 | * Integers | ||
| 19 | * Floats | ||
| 20 | * Strings (binary 8-bit strings, perfect for UTF-8) | ||
| 21 | * Booleans | ||
| 22 | * Nulls | ||
| 23 | |||
| 24 | # License | ||
| 25 | |||
| 26 | GATS is distributed under a new-BSD type license. This is fairly permissive, | ||
| 27 | but if you require other licensing options feel free to contact us. | ||
| 28 | |||
| 29 | # Uses | ||
| 30 | |||
| 31 | GATS was originally intended for use in internet protocols, and is great for | ||
| 32 | that purpose. Used correctly it allows you to create efficient yet highly | ||
| 33 | malleable protocols with very little extra effort. | ||
| 34 | |||
| 35 | However, you can certainly use it for other purposes such as serializing data | ||
| 36 | for storage, and inter-process communication. In fact, with the different | ||
| 37 | language bindings available, sometimes GATS may be one of the easier ways to | ||
| 38 | allow, say, a PHP web service to communicate with a custom C++ executable or | ||
| 39 | python program. | ||
| 40 | |||
| 41 | # Languages/Libraries Supported | ||
| 42 | |||
| 43 | At the moment we actively maintain libraries for C++, java, php, and python. | ||
| 44 | Other languages and libraries are welcomed. Here's a little info on each | ||
| 45 | target directory: | ||
| 46 | |||
| 47 | * ''c++-libbu++'' - The original libgats implementation. Works using libbu++ | ||
| 48 | data types and streams. You need to have libbu++ and Xagasoft build in | ||
| 49 | order to build this version. | ||
| 50 | * ''c++-qt'' - A version written using Qt data types. This version builds using | ||
| 51 | qmake, so if you're using Qt you already have everything you need. Also | ||
| 52 | features handy signals & slots to make event driven networking even easier! | ||
| 53 | * ''java'' - A library using the Java native interfaces, everything looks and | ||
| 54 | works exactly how you would expect it should. There is a Xagasoft Build | ||
| 55 | script to build a jar file, but it's simple enough that a single javac | ||
| 56 | command can build it all, or just import the code into your project directly. | ||
| 57 | This java version has been used on desktops and android devices. | ||
| 58 | * ''php'' - There are two libraries for working with php, the first defines a | ||
| 59 | set of classes for fine control over the format, sometimes this is necessary | ||
| 60 | as php's types are a little loose. The second simply uses php native types | ||
| 61 | like array() as data transport. The second option is usually the much easier | ||
| 62 | to use, but doesn't always get the encoding correct for all inputs. | ||
| 63 | * ''python'' - These work like other serialization mechanisms in Python like | ||
| 64 | pickle, json, shelve, and marshal. It exposes the functions load, dump, | ||
| 65 | loads, dumps, and also the handy helpers recv and send for working with | ||
| 66 | sockets. The Python implementation returns and transmits native Python | ||
| 67 | data types, which makes life pretty easy. To use this version simply copy | ||
| 68 | gats.py to your project. | ||
| 69 | |||
| 70 | # Basic Operation | ||
| 71 | |||
| 72 | The way GATS works is dictated by the format, so it works similarly in every | ||
| 73 | implementation, although they each have slightly different mechanics. When | ||
| 74 | encoding GATS you always encode each object in it's own "GATS packet." A GATS | ||
| 75 | packet has a very simple header which includes the size of the packet to make | ||
| 76 | parsing fast and efficient. | ||
| 77 | |||
| 78 | Each packet can contain a single root object. It can be any type, but for most | ||
| 79 | protocols a dictionary is a great choice for the root object. | ||
| 80 | |||
| 81 | The format is designed to make it very easy to work with various encoding, | ||
| 82 | packing, and encryption systems. The reader, by default, will skip all leading | ||
| 83 | zero bytes that come before a valid GATS packet, and will stop processing | ||
| 84 | precisely at the end of a valid GATS packet. | ||
| 85 | |||
| 86 | Skipping leading zeros makes it easy to work in environments where padding may | ||
| 87 | be required. You can use the simplest of all padding schemes (pad with zeros) | ||
| 88 | and it will work seamlessly with GATS. | ||
| 89 | |||
| 90 | Since the reader always reads exactly the number of bytes it needs, it's very | ||
| 91 | easy to embed GATS packets in other streams, or read them sequentially as fast | ||
| 92 | as you can from a socket. | ||
| 93 | |||
| 94 | ## A Note About Strings | ||
| 95 | |||
| 96 | All strings in GATS are simply sequences of 8-bit bytes. There is no | ||
| 97 | overarching encoding that is dictated by the format. When using GATS it is | ||
| 98 | good to specify how you are encoding your text data, we recommend Unicode. | ||
| 99 | There is a possibility that a future version of GATS will include a separate | ||
| 100 | Unicode String data type, but for now it's important to remember this. | ||
| 101 | |||
| 102 | For this reason, we also recommend making the keys in all dictionaries 7-bit | ||
| 103 | UTF-8 compatible ASCII/Latin1. This isn't required of course, but it makes | ||
| 104 | things a bit easier. | ||
| 105 | |||
| 106 | # Speed vs Size | ||
| 107 | |||
| 108 | GATS objects are, on average, smaller than storing in other binary formats, and | ||
| 109 | can be much smaller than textual formats by virtue of storing only as many | ||
| 110 | bytes as necessary for integers and floats. This also means that GATS requires | ||
| 111 | more processing than fixed field binary formats, but interestingly not quite as | ||
| 112 | much as text formats like json. The processing we do on floats is actually | ||
| 113 | roughly comparable in many ways to text processing, although with fewer steps. | ||
| 114 | |||
| 115 | |||
