diff options
Diffstat (limited to '')
-rw-r--r-- | README.md | 115 |
1 files changed, 115 insertions, 0 deletions
diff --git a/README.md b/README.md new file mode 100644 index 0000000..f5302a8 --- /dev/null +++ b/README.md | |||
@@ -0,0 +1,115 @@ | |||
1 | # Generalized Agile Transport System (GATS) | ||
2 | |||
3 | GATS is a data encoding format, as well as a set of libraries for reading and | ||
4 | writing data in that format. It's a structured format reminiscent of a binary | ||
5 | json, but with some extra cool features. | ||
6 | |||
7 | * All numbers (integers and floating point) are stored in an arbitrary | ||
8 | precision format which takes up the least amount of space it can. | ||
9 | * Unlike textual formats (like json, xml, etc.) floating point numbers are | ||
10 | stored exactly, the exact number you write in is the exact number you read | ||
11 | out. | ||
12 | * All formats stored are machine independent, you can read and write them on | ||
13 | any architecture, any endianness and they will always be the same. No more | ||
14 | encoding worries. | ||
15 | * There are a number of data types you can use to make up data structures: | ||
16 | * Dictionaries | ||
17 | * Lists | ||
18 | * Integers | ||
19 | * Floats | ||
20 | * Strings (binary 8-bit strings, perfect for UTF-8) | ||
21 | * Booleans | ||
22 | * Nulls | ||
23 | |||
24 | # License | ||
25 | |||
26 | GATS is distributed under a new-BSD type license. This is fairly permissive, | ||
27 | but if you require other licensing options feel free to contact us. | ||
28 | |||
29 | # Uses | ||
30 | |||
31 | GATS was originally intended for use in internet protocols, and is great for | ||
32 | that purpose. Used correctly it allows you to create efficient yet highly | ||
33 | malleable protocols with very little extra effort. | ||
34 | |||
35 | However, you can certainly use it for other purposes such as serializing data | ||
36 | for storage, and inter-process communication. In fact, with the different | ||
37 | language bindings available, sometimes GATS may be one of the easier ways to | ||
38 | allow, say, a PHP web service to communicate with a custom C++ executable or | ||
39 | python program. | ||
40 | |||
41 | # Languages/Libraries Supported | ||
42 | |||
43 | At the moment we actively maintain libraries for C++, java, php, and python. | ||
44 | Other languages and libraries are welcomed. Here's a little info on each | ||
45 | target directory: | ||
46 | |||
47 | * ''c++-libbu++'' - The original libgats implementation. Works using libbu++ | ||
48 | data types and streams. You need to have libbu++ and Xagasoft build in | ||
49 | order to build this version. | ||
50 | * ''c++-qt'' - A version written using Qt data types. This version builds using | ||
51 | qmake, so if you're using Qt you already have everything you need. Also | ||
52 | features handy signals & slots to make event driven networking even easier! | ||
53 | * ''java'' - A library using the Java native interfaces, everything looks and | ||
54 | works exactly how you would expect it should. There is a Xagasoft Build | ||
55 | script to build a jar file, but it's simple enough that a single javac | ||
56 | command can build it all, or just import the code into your project directly. | ||
57 | This java version has been used on desktops and android devices. | ||
58 | * ''php'' - There are two libraries for working with php, the first defines a | ||
59 | set of classes for fine control over the format, sometimes this is necessary | ||
60 | as php's types are a little loose. The second simply uses php native types | ||
61 | like array() as data transport. The second option is usually the much easier | ||
62 | to use, but doesn't always get the encoding correct for all inputs. | ||
63 | * ''python'' - These work like other serialization mechanisms in Python like | ||
64 | pickle, json, shelve, and marshal. It exposes the functions load, dump, | ||
65 | loads, dumps, and also the handy helpers recv and send for working with | ||
66 | sockets. The Python implementation returns and transmits native Python | ||
67 | data types, which makes life pretty easy. To use this version simply copy | ||
68 | gats.py to your project. | ||
69 | |||
70 | # Basic Operation | ||
71 | |||
72 | The way GATS works is dictated by the format, so it works similarly in every | ||
73 | implementation, although they each have slightly different mechanics. When | ||
74 | encoding GATS you always encode each object in it's own "GATS packet." A GATS | ||
75 | packet has a very simple header which includes the size of the packet to make | ||
76 | parsing fast and efficient. | ||
77 | |||
78 | Each packet can contain a single root object. It can be any type, but for most | ||
79 | protocols a dictionary is a great choice for the root object. | ||
80 | |||
81 | The format is designed to make it very easy to work with various encoding, | ||
82 | packing, and encryption systems. The reader, by default, will skip all leading | ||
83 | zero bytes that come before a valid GATS packet, and will stop processing | ||
84 | precisely at the end of a valid GATS packet. | ||
85 | |||
86 | Skipping leading zeros makes it easy to work in environments where padding may | ||
87 | be required. You can use the simplest of all padding schemes (pad with zeros) | ||
88 | and it will work seamlessly with GATS. | ||
89 | |||
90 | Since the reader always reads exactly the number of bytes it needs, it's very | ||
91 | easy to embed GATS packets in other streams, or read them sequentially as fast | ||
92 | as you can from a socket. | ||
93 | |||
94 | ## A Note About Strings | ||
95 | |||
96 | All strings in GATS are simply sequences of 8-bit bytes. There is no | ||
97 | overarching encoding that is dictated by the format. When using GATS it is | ||
98 | good to specify how you are encoding your text data, we recommend Unicode. | ||
99 | There is a possibility that a future version of GATS will include a separate | ||
100 | Unicode String data type, but for now it's important to remember this. | ||
101 | |||
102 | For this reason, we also recommend making the keys in all dictionaries 7-bit | ||
103 | UTF-8 compatible ASCII/Latin1. This isn't required of course, but it makes | ||
104 | things a bit easier. | ||
105 | |||
106 | # Speed vs Size | ||
107 | |||
108 | GATS objects are, on average, smaller than storing in other binary formats, and | ||
109 | can be much smaller than textual formats by virtue of storing only as many | ||
110 | bytes as necessary for integers and floats. This also means that GATS requires | ||
111 | more processing than fixed field binary formats, but interestingly not quite as | ||
112 | much as text formats like json. The processing we do on floats is actually | ||
113 | roughly comparable in many ways to text processing, although with fewer steps. | ||
114 | |||
115 | |||