Most systems build today in corporate environment use very similar architectures and often rely on standard infrastructure like databases, web servers and application servers. Developers use a multitude of open source Frameworks, stick them together and can quickly deliver functionality, without spending too much time on the technical aspects, the “plumbing”.
Capital markets, telecommunication, online gaming, social networks and other industries face technical challenges where those standard tools find their limits and alternative architectures and technics are required. One of the challenges developers face when they build such system is the lack of standard reusable components and most of the time have to re-implement everything from scratch.
Products and services
The foundation will focus on delivering shared implementations of core services that Banks find themselves implementing time and time again. For example the foundation could deliver enterprise grade reference data platforms, market data platforms, OMSs, algorithm containers, trade repositories, grids for risk, pricing, etc. Some of those services could even be offered as a shared infrastructure, on the cloud.
Layered architecture
High level products and services is the end goal and it will take some time to get there but we believe the foundation should provide value as soon as possible. This is a very good reason to adopt a layered architecture and break down those high level components into low level libraries and release them earlier in the process.
The layered and componentized approach offers several other advantages:
- Users are free to use the complete solution or pick only what they need,
- Framework and libraries are easily adopted; whereas infrastructure components require support, training, etc.
- Contributors and maintainers have their respective area of expertise and can focus on the components they are interested in, without dealing with the complexity of the overall system,
- Banks and other partners will donate software to the foundation. It would be very complex to compare similar high level components, like a CEP solution for instance, and pick the best one. Instead we can compare the low level components, pick the best ones and combine them to build higher level constructs.
License
We plan to use a very open license, something like Apache 2.0 or MIT.
The diagram below highlights some of the components the foundation could undertake to build, from the bottom to the top.
| Component | Layer | Description |
| Disk I/O: Log appender | Libraries | Logging should not be a bottleneck in a high performance system. Standard Frameworks like log4J or log4net have not been designed for high throughput, low latency system: they create too much garbage (GC pressure) and are often source of contention in the system. This library should offer an in-proc or out-proc appender with extremly low overhead on the system. |
| Disk I/O: Transaction journaler | Libraries | Transaction journaler is a high performance facility used to store all events in a durable form, so that they can be replayed should anything go wrong. Journaling can be very useful in a deterministic system to reproduce a product issue in a test environment: grab the production journal, replay in test with a debugger attached, enjoy! |
| Marshaling | Libraries | Langage neutral, platform neutral mechansim to marshal and unmarshal data. Marshalers are generated at compilation or at runtime and offer absolute performance. Schemas, validation and versioning supported. |
| Parsers | Libraries | Highly optimized parsers for text based messages. |
| Data collection | Libraries | Hardware friendly data collections: data locality and minimized cache misses are key to high performance data strctures like hashtables, trees, etc. For managed environment it is important as well for those data structures to be friendly with GC algorithms. |
| Concurrency toolkit | Libraries | TODO |
| IPC | Libraries | Langage neutral, platform neutral mechanism to communicate between two or more processes. |
| Serialization | Libraries | TODO |
| Event service container | Platform | TODO |
| CEP | Platform | TODO |
| Monitoring and management tools | Platform | TODO |
| User interface: widget container | Platform | Host application modules in isolation, in different processes. Multiple development teams can develop, test and release modules independently. Modules can expose a user interface or be face-less and run as background services (deamon). |
| User interface: Deployment | Platform | Service responsible to install and keep up to date the widget container and selected modules. Update of modules happens in background, as a side by side installation and transparently for the user. |
| User interface: Window manager | Platform | Provides docking functionalities and workspace management for the different windows of the widget container. |
| User interface: UI Controls | Libraries | TODO |
| OMS | Product and services | TODO |
| UI Widgets: Blotter | Product and services | TODO |
| Calendar and dates | Product and services | TODO |
| Reference data platform | Product and services | TODO |
| Trade repositories | Product and services | TODO |

My thoughts: http://etrading.wordpress.com/2012/05/23/lodestone/