Tlaloc

Chapter 2 : Advanced concepts

Pushing things a bit further has always been my ultimate goal, and this is where it starts to get juicy.

A couple of advanced concepts

• Administrative web interfaces
• Message Queue
• Tri-State results and Boolean results
• Crypto object (requires libsodium), 2016
• Infrastructure Recipes
• Combining Recipes and Message Queueing to obtain a fully automated server blob.
• New PHP7 support. Yeah baby!

Administrative web interfaces

A set of administrative interfaces are available under a website package within the Tlaloc system, which can be installed to monitor and queue up Messages in the Message Queue.

Message Queue

The original intention was to program a Peer2Peer cloud for the messaging part, and this idea is not off my mind still. But in the meantime, the system includes a database-driven Message Queue system, matched with a Queue class object.

Back in 2000, it was called a Message Queue, then the moniker changed to Enterprise Message Bus, and nowadays more commonly known as a Service or Asynchronous Messaging Bus.

Feb 2019 update; Microsoft has renamed it's message queue to "Service Bus" in Azure, some users might be familiar with that terminology.

The purpose of the Message Queue is to distribute Queued orders for configuring Users, Files, Folders, Configurations, anything that can be controled by Tlaloc and other service/backend stacks.

Most commonly used to decouple applications and services from each other, and as a reliable and secure platform for asynchronous data and state transfers. Data is transferred between different applications and services using messages which are stored in a central database, in an accessible format for programmers to tinker with.

The Queueing API includes a Reader methodology for reading one Queued item at a time, a dependency (parent-child relationships) is also built into the system for chaining elements together.

In order to Queue up items, the programmer declares a chain of Queue commands through the API, then compiles the Queued commands which saves the Queue on the central database system. The queued items are ciphered using a 2 way public/private key system allowing the administrator access to the monitoring of Queue Messages, and making sure only the target node can read and process the Queued Message.

Tri-State results vs Boolean results

During my implementation of the Queue system for managing users and websites, I quickly discovered I was restarting a number of services too repeatedly, and for nothing. So I reworked all the objects to use a Tri-State return value. The beauty of the system resides in its compatibility with Boolean results (in an indempotent system exceptionally).

Follows are these 3 self-explanatory states. They are defined as global constants when you include the Tlaloc.php file in your script. Use them as is in your code.

• STATE_SUCCESS; indicative of a successful operation (ie; something was done with success), evaluates to TRUE as a boolean
• STATE_NOTHINGTODO; indicative that nothing had to be done (and usually it means it was double-checked at this point), evaluates to TRUE as a boolean
• STATE_ERROR; indicative that an error occurred while attempting to modify something, this evaluates to FALSE as a boolean

Crypto object

Developped in the winter of 2016-2017, the Crypto class is a wrapper around php_libsodium, which is a wrapper around libsodium, itself a wrapper around Sodium.

I'm now making this class part of the Tlaloc package because the Queue Messaging system depends on it. And quite honestly, I think it'll become very handy to be able to eventually encrypt files, and manage keys properly.

The advantages to using the Crypto object are numerous. For one, it introduces the concept of SafeStrings, a form of variable storage which cannot be dumped during a core dump, or through a debugger. These are extensively used in the Crypto object to protect decrypted keys and strings.

Another advantage is the strong cryptographic routines which can be employed to protect sensitive information in your recipes, such as PASSWORDS !

The Crypto module also includes the functionality to generate strong Argon2 type password hashes (ie; one way hashes that require a lot of computing power to compare), but that's best used in a web ui scenario for a login process.

The Crypto module also includes HKDF2 methods for deriving additional keys, secure hash validation methods, password to key derivation routines, Merkle Tree structures (to be enhanced) and much more.

Sodium's reputation at stake; I'm aware of the dissent caused by Sodium in the community, it's sources being contested as "opaque" compared to other cryptographic libraries. But the reality is, its currently the only functional package as of today under PHP. I personally had to go through great lengths to compile Sodium in my PHP packages at work.

Infrastructure Recipes

Using all the Tlaloc tools over the years I've developed a decent amount of recipes through my daily work. From this I'm distilling a number of useful recipes for installing all kinds of packages.

• Chrooted OpenHTTPd with optional ClamAV, PHP-fpm
• Unbound/Bind DNS server setups
• MariaDB master-slave setups, cluster setups and DB kickstarters
• Firewalls, bastions, bridges, VPN nodes
• Distributed file services
• OpenBSD VM management; different VM recipes, start and install and push Tlaloc to VM instances.
• Kernel compiling; compile a fresh OpenBSD (for now) kernel and a set of Ports from a cronjob! (soon through a VM to make things easier?)
• Monitoring recipes (disk, cpu, memory, network, ARP tables, routes, firewall stats, etc...)
• Reporting recipes (hardware asset auditing, brand, makes, models, features, quantities, logs, etc...)
• Backups, Replication, bit of Conversions too
• DHCP configuring
• Serial - switches through Cisco IOS and Expect ;) get a list of all serially connected devices, obtain ARP tables for those switches capable of producing them, running config (read & write), switch resets, etc...

Combining Recipes and Message Queueing

So, what do you get when you combine Server Configuration Recipes and Service Automations through a Secure Message Queueing protocole ? According to me, the perfect automation engine for turning complex hardware into a real blob of lego blocks.

If this vision doesn't make sense to you, allow me to explain;

Each Node can auto-bootstrap with the Tlaloc eco-system at boot time.

Nodes advertise their capacities to the Tlaloc eco-system, the Administrator can fine-tune what type of services are required in the eco-system and their redundancy requirements

Nodes then decide on their own what services to activate according to the required redundancy

Each Node can run a cronjob executing the Message Queue Polling Service

Each Message is properly encrypted (protecting potential sensitive information)

A Node can communicate globally with all the other nodes (still in encrypted fashion), or with specific nodes, or with a user upon request (with proper crypto keys)

A User can inject commands in the form of Queued Messages through a simple web interface, or by integrating the API programatically

The current libraries cover pretty much everything involving network infrastructure, security, installation of packages, user and service management, everything a corporate network requires to deploy services (or a hosting provider)

The Queue Messaging system can be used to dispatch all kinds of jobs to listening cronjob daemons. Languages are no barrier, as long as they can execute a command line script to obtain a message, and read deposited files.

Therefore... deploy physical nodes in your blob, and they auto-configure to the requirements of the day. If your nodes detect trouble, they'll launch the necessary backup nodes THEMSELVES. (This works better in a virtual limitless environment, like Amazon for example, or when you have a bunch of spare hot-servers)

Another un-realized benefit from all this, in Kopel's workplace, is that this implementation greatly reduces the complexity for non-technical users to manage customer services deployed on our servers. As it stands right now, a Project Manager can issue and create his required users, configure a website, install required applications as per customer specifications and rest assured that all the required dependencies (SSH, SFTP, Folders, permissions, Firewall, Samba users, Project folder) are created in a timely fashion.

A new benefit, arising from the realisation of the extents of this project is that we went beyond our previous philosophy of using commodity hardware succesfully. This iteration allows us to literarily turn the redundancy issues into a non-issue. Just plug in a new BLANK server, and the eco-system takes care of installing and dedicated the required services from the get-go. Of course, this total integration doesn't exist as of today, but we are definitely getting closer to this (this year for sure).

So, given all the above benefits of the combination of the above concepts, follows what we successfully integrated so far:

• SSH Service Nodes : Create,Remove,Verify Users, reset Passwords, synchronize authorized_keys (to the repository) and reset authorized_keys (if not user-updated optionally)
• FS Nodes : Move, Encrypt, Email, Diff, Publish, Scan and Replicate files
• Samba and its user management; Create, Check, Reset Password, Delete User, Create Share
• PF and per-user anchors using a proprietary SSH tunnel shell
• DNS services : Add/Delete domain, Edit Zonefiles
• SQL databases: user management, database management

In Kopel's world, there exists another little (secret) project which is currently integrated in our Queueing system, but it's not meant for public knowledge. (This is just so Kopel staff understand why the element is missing from this list. Hihihi)

What's in the future for Tlaloc ?

Well, through these late epiphanies I've come to the realisation that Kopel had developped (through my programming skills I guess), a great File Management interface. I'm not too proud of it's current looks (exceptionally the Internet-public side), but I'm confident that with a little facelift, we can turn this solution into the best secured file storage solution.

First offs, Kopel has never suffered a data breach affecting customer data. Ever.

Secondly, I am confident that with my late works on cryptographic implementation, and with proper key management, we can turn this data storage into a valuable proposition for our customers, and would-be customers.

Thirdly, a lot of our customers already use our secure data file transfer application in their relationship with us, we've been running this type of service for more than 16 years now, and constantly improving on it, we are confident that our solution resides at the top of the competitive pack security-wise.

Finally, as a data-treatment provider, Kopel already offers services attached to stored data files in our environment. Now that we're seeing customers send us files from Google Drive, we thought it wise to extend our more secure offering to our preferred clients for their daily secure data storage needs.

So, I intend to focus a bit more on packaging the front-end interfaces (currently hidden away in the Extranet behind a password) of our Secure File Manager, design an auto-enrollment form for people wishing to store data files in a Secure Escrow Storage Service and integrate the whole with Tlaloc to manage user accounts. Our current feature set makes it easy to setup a truly resilient service, hosted on our own equipment, and secured behind 2 layers of unbeatable filters.

And, this would be a great opportunity for Kopel to turn an obligation into a financial asset. We currently do not charge for any of our equipment's use (well barely in exceptional cases), so figuring new ways of offering new innovative services that can generate a profit that can help pay for all our equipment is now my duty. (In a way that the executive team will agree of course!)

So stay tuned for these developments! It could be the beginnings of a great online app.

PHP7 Support!

Some of the Tlaloc object have already suffered through the gruesome hardcore revising for PHP7, under a new IDE, giving rise to a new code quality.

It is my intention to complete this PHP7 upgrade for all objects, because it's a lot more nifty, and some of the new features make for much cleaner coding.

Some people might feel uncomfortable with running PHP7 on production systems, so the whole programming is still PHP5.6 compatible. ;)

Upcoming feature, June 2017:

An autoloader is planned for the next release. Current development on my other projects already use that methodology extensively, and with cleaner code and better error handling. So it's a given that this feature is coming over. I also see the possibility of extending the autoloader in the Tlaloc Nodes to put the OS-toggling features in it. (They are currently programmed in the Tlaloc.php include file, copy&paste really...)