There are many core application protocols that are used in various type of processing such as HTTP used to transfer hypertext and hypermedia over web information systems, SMTP used for e-mail transmission between devices, RTP used for delivering audio and video over networks, and FTP used to move data from one server to another one.
Protocols like HTTP, SMTP and FTP are depending on TCP protocol which is reliable and connection-oriented, while RTP is depending on UDP protocol which is unreliable and connectionless-oriented.
Reliable file transfer is a vast field of work. However none of the current protocols provides enough robustness, reliability and flexibility, except proprietary and patended ones.
Since its early days, SmartJog has developped its core file delivery service based on a protocol as a fast, reliable and secure protocol called RBC (Reliable Bit Cast).
Created in 2004 by SmartJog, RBC is distributed under the terms of the GPL where SmartJog is the copyright holder.
The current version deployed in production is the 4.3.7.
RBC is a network protocol allowing reliable file transfer over any IP network, such as the Internet, Ethernet LANs, satellite links, etc.
RBC can use TCP, UDP, or UDP over IP over TCP :
- TCP makes RBC benefit from the reliability of this transport layer
- UDP enables multicast or broadcast transfers
UDP over IP over TCP allows interoperability with generic IP encapsulators (such as IP/DVB encapsulators used for satellite links)
Operating model
A RBC transfer occurs between a sender and a receiver.
In the RBC terminology, a transfer is called a “stream”.
Both the server (RBC Send) and the client (RBC Receive), runs as UNIX daemons.
These two daemons runs on the same host, in order to send and receive files to/from the same host at anytime.
Robustness
One of the main value of RBC is that it has been elaborated with robustness in mind ; because network failures matter, software failures matter, in short real-world matters.
If the data channel connection breaks, RBC Send tries fo reconnect to RBC Receive to resume transfers.
RBC Receive sends continuously its status to RBC Send, so one can monitor precisely the situation from the server.
RBC Receive and RBC Send save their global state at regular intervals. It means RBC Receive and RBC Send are able to resume interrupted transfers in case of kernel crash, power outages, hard-reboot, and persistent network failures.
Large File Support
The file size to be transferred is encoded in a 64-bit integer. So the maximum file size supported by RBC protocol is 264 bytes, or about 18 ExaBytes.
In practice RBC has already been used to transfer files above 1 TeraByte.
You can be comfortable with the use of UHD files, whatever the frame raster, codec and bit rate choice.
Automatic Retransmission
An Automatic Repeat Request (ARQ) mechanism is implemented by RBC.
The receiver periodically sends NACKSs to the sender, NACKs indicate which data packets are missing so that the sender knows which packets need to be retransmitted. If no packets have been lost, a special empty NACK packet is sent: this is a zero-NACK packet.
Data Corruption Detection
At the time of RBC Send is sending a file, it computes its MD5 check-sum (or message digest) on-the-fly. RBC Receive does the same compute on its own side.
When RBC Send reaches the end of file, it sends the final checksum, RBC Receive compares it with its own checksum, and depending on the result, a success or error message is sent back to RBC Send.
Unless an important network outage, the process goes through until the delivery check-sum is identical to the source one.
SmartJog does garanty 100% of file integrity.
Encryption
RBC implements AES data encryption at high throughput (over 1 Gbps).
Another security layer is then added to your valuable assets.
Interrested by the SmartJog plateform and services ?
Contact our sales team or request for a demo