Just finished the Server-Client loop, as well as overhawling the parser binary. We are ready to SHIP!

Added the data generation, request creation, and response creation to the client and server. You can see a demo below. Not a giant change, just integrating the the new functions into the Client and Server.

IM FREEEEE

I FINALLY finished writing the write_data() function. Now full packet generation, with a request, and a reply is complete!

Litterly the only thing left to do is make the ui. I AM DONE WITH THE BACKEND!!!!!

I have included a handy guide to the output bellow. Instead of another picture of my terminal, you guys get to enjoy some color coded google docs!

Im just going to appologize now, this devlog is not gonna be that great.

I have been working the past few days on packet generation. Specifically on requesting data from a server, and writing that data to a packet as a server. As you can see by the time, this is much harder than it may seem. Though I did predict this a while ago.

When the client request data from the server, a function is invoked that configures the group (data type; digital or analog) and the variation (presentation; packed format or with extra flags).

When the server responds to this request, it invokes a function (technically 2) that refreshes the data it has (currently using a random number generator) and writes that to a packet. The packet is mostly a clone of the request, just with the source and destination switched, and the data added.

Currently, I have both functions completed (or almost completed), but I still need to add crc to the reply form the server, and fix the many bugs that have arrisen from this change.

Bellow I have an example request, no reply becuase the function is currently inoperable :(

05 64 0F C4 01 00 00 00 C8 E2 
C1 C0 01 01 01 10 F7 03 00 0F 
00 00 

This request is asking for data from the server with address 1, data type 30 (digital), and the data points 3-15

Once again, very sorry for the poor quality dev log, but I just wanted to get an update out before I dug myself into an even deeper hole.

See you guys later!

Been a busy 4 days.

Worked on mostly modularity for the program, making sure that I am not repeating code that can be a function as part of a larger library

The 2 libraries I implemented were the common/ and network/ libraries. These are used in both the client and server. Common includes functions like setting up connections, sending packets, receiving packets, interpreting them, and polling them for data

The network library handles connecting to a server/client, and others.

I also finally added the ability for the client (main stations/ems) to multiple servers (outstations/rtu). You can see a demo of all this working in the video.

That’s it for now, goodbye!

More work on the logging system. Realized that we can use a global var with an init function to remove the need for each log call to pass the file pointer to the log

Previous Method;

// main
log_err(errno, "client/main", fptr);

The fptr is a FILE* varible that points to the logging file

New Method;

// log.c
static FILE *log_file = NULL;

int log_init(const char *file_path) {
    log_file = fopen(file_path, "a");

    if(log_file == NULL) {
        log_err(errno, "log/log_init");
        return -1;
    }
    return 0;
}

// main
log_init("log/log.txt");

log_err(errno, "client/main");

The new method means all you have to pass is errno and the location of error.

Super Simple!!!

I also updated both client and server with this system.

Bye

Small addition. Made log_info() a variadic function, like printf(). This means you can now do the same formatting like printf("hello %s", "world"). This allows for easier logging to file and console, as both are just an arg in the funciton call!

Yay!!!!

Finished adding the logging library I mentioned in the last post. It supports Colored text for warning and errors, time stamps, and better formatting.

Ironically, the hardest part about this was figuring out how to use ANSI escape codes to support the colored text. This Medium article fromVitor Matias was super helpful!

i also realized most people dont wanna read walls, so my updates will be smaller form now!

DevLog 15

Been busy the past couple of days with some school, so sorry I haven’t posted sooner, but I have also built out a couple more features.The Main additions are;

1. Reusable frequent packet structures (acknowledge, negative acknowledge, reset link)
2. Server Client loop
       1. Ability to choose packet type to send to server (through TUI) in client
       2. Server responds, and client is able to display response
3. Better error handling, with more checks for errno and printing to log and console
4. Better logging through file rather than just console

The logging is proably my favorite new feature. It saves previous runs and even annotates the time and date the run took place. The error info is also much more helpful, and more often my program tells me why it crashed, instead of just ending or segfaulting

client [main]: Connection refused

^^^^ This now happens when you try to run the client with out a server on the configured adress or port

Errors I ran into

There actually wasn’t that much stuff going wrong, just took me pretty long as I had to edit a lot of functions and flows.

A big issue i did run into was adding select(), also know as polling to the socket. Basicly, my code will loop checking the socket and parsing the data in it. But, a problem occurs if there is no data in the socket waiting.

Polling allows my code to check if there actually is data, and if not wait/move on to the next task. But, I had incorrectly configured the function (the documentation for this function is especially confusing…), and this lead it to always report data in the socket. This then lead to errors with trying to map that data into a struct, and it just lead to catastrophy.

Immediate Plans

Have some stuff I’m going to / want to try and start before the next devlog;

• Logging Functions to simply logging to console and file
      * Something like log_note(), log_warn(), and log_error()
      * Currently just writing both fprintf() and perror()
• Add functionality to digital and analog requests    * Need to add packet structures and creator functions for both
• Seperate network functionality like socket(), send(), etc… into its own library fucntions
• Seperate shared code between server and client into common library

Media

I thought I would include this section to explain some of things in the attached videos. These show all the features I implemented, including the client server loop, logging, and error handling

Video 1

This just shows the current loop between the client and server, where the client sends a packet, a reset link packet in this case, and the server responds, in this case with a simple acknowledgement.

It also shows how both programs log information about their runs in an assocaited text file. Btw, if you have some sugestions for making this better, please leave a comment. This is just about the simplest way to do logging, so other ideas are appreciated!

Video 2

This shows an example of custom ports, error handling, and more logging. Both the client and server support custom ports (and server adresses in the case of the client) passed in as a launch argument.

You can see when the client trys to connect to a server that either doesn’t exist on the current port, or isn’t active, an error message is displayed into the console and logged in the file.

Alternatively, when both use the same port, the connection succeds, that port is logged as well.

Wrap Up

Lots of stuff to do still, and I am super excited to get started. Thank you to the few of you who have been following development since the beginning, and hello to anyone new!

Manual Packet Creation and More!

Just finished adding the ability to create a packet based on parameters rather than requiring a premade packet.

Manual Packets

Basicly, instead of needing something like ,

uint8_t input[] = { 
    0x05, 0x64, ...
};

dnp3p_st packet_s = mkPacket(input);

You can now just use a function and pass in parameters. Then, since this is just a fancy memory structure, a simple memcpy into an appropriately sized array will give you both! (The raw packet, and the structure)

uint8_t packet[30] = {0};

 // did some renaming, uses dnp3Lib for library funcs now
dnp3p_st packet_s = dnp3Lib_mkPacket_manual(1, 0, 30, 2, 0, 10);
// params are: destination, source, group, variation, startIndex, and stopIndex

memcpy(&packet, &packet_s, 30);

This will give you basicly the same thing as the first example, meaning you can create a packet based on a discription, rather than a discription based on a packet.

This will be very useful when creating responses from to the client from the server, or vice versa.

Should be noted that this is technically not very complete, but it is just an example for now.

Pre-made Packets

On the topic of creating a packet based on a discription, I also created some functions to create packets for common messages, including;

• Reset Link
  • Required when creating a new connection
• Acknowledgement
• Negative Acknowledgment

These will be useful when making connections, and sending responses to requests and replies.

Wrap Up

Still got a ways to go in terms of completion, but I’m hoping to ramp up my speed, been a little busy the past couple weeks.

Added the ability to pick a custom socket or server ip. Also actualy pushed becuase I forgot last time

Just finished getting the client server connection working.

Next thing to do is to interpret the packet on the server side, then create a reply based on the request.

Dev log 11?

I just pushed some changes and merged main and dev. This update I basically just made some fixes that should make the parser more accurate overall, but there are still a few lingering bugs.

The main change I made was adding a system to clean the packet of any CRC bytes from within. Normally, a packet contains 2 bytes of CRC spread around it, with 2 after the end of the header, and after that 2 after every 16 bytes.

This change will allow my data handler to not accidentally ingest CRC bytes, which would mess up the spacing of each data segment, and result in randomness.

I also added a function to calculate the size of a packet based on the length information stored within it, rather than the physical memory size of it. This is better, as I can call it from any function and not have to pass in the size as a param.

Overall, I am around 90% done with the backend, and just have a couple bugs to fix.

I have attached a screen shot of some of the new features.

Added parsing for more Objects

Has not been very long but I am back with an update. I just finished adding in parsing for more object types, including

• Group 1
  • Varriation 1
  • Varriation 2
• Group 30
  • Varriation 1
  • Varriation 2

I’ll also try and make more frequent posts here!

DNP3 Parser Update - Understanding a DNP3 Packet

This week I finished parsing DNP3 Group 1 Variation 1, the first object type supported by the parser. That means the parser can now decode binary input status values from real DNP3 traffic instead of just stopping at the object header.

I also got some practice using GDB to debug my programs! Super useful as I ran into some segmentation faults that would have been impossible to fix without it!

How a DNP3 Packet Works

DNP3 (Distributed Network Protocol 3) is an industrial control protocol used by electric utilities, water systems, and other critical infrastructure. Communication happens between a Master (SCADA server) and one or more Outstations (RTUs, relays, PLCs, etc.).

A packet is made up of several layers:

1. Data Link Layer
2. Transport Layer
3. Application Header
4. Object Header(s)
5. Object Data

Data Link Layer

The packet always begins with the start bytes:

0x05 0x64

These identify the frame as DNP3.

The rest of the link header contains:

• Length
• Control byte
• Destination address
• Source address
• CRC

This layer is responsible for moving frames between devices and detecting transmission errors.

Transport Layer

The transport header is only one byte long, but it’s important because it:

• Marks the start and end of fragmented messages
• Provides a sequence number so fragments can be reassembled

Application Layer

The application header tells the receiver what operation is being requested.

Examples include:

• Read
• Write
• Response
• Confirm

It also contains application control flags and an application sequence number.

Object Headers

After the application header come one or more object headers. These tell the parser what type of data follows.

An object header contains:

• Group number
• Variation number
• Qualifier

For example:

Group 1 Variation 1

represents packed binary input values.

The qualifier then specifies how many objects are present and how they are indexed.

Object Data

Finally comes the actual payload.

For Group 1 Variation 1, each bit represents the current state of a binary input:

• 1 = ON
• 0 = OFF

The parser reads these bytes and expands them into individual points that can be displayed or analyzed.

Completed Sections

• Data Link Layer parsing
• Transport Header parsing
• Application Header parsing
• Object Header parsing
• Group 1 Variation 1 object parsing

Next up is adding support for additional object groups and variations so the parser can decode more real-world DNP3 traffic.

Transport Header, Printing Updates, and Bug Fixes!

Its once again that time. This week I worked on the object header, updated the printing on the DLC parsing, as well as fixing a couple big bugs.

Transport Header

The transport header consists of 5 parts, and its main job is to explains the format, and ordering of the data in the packet. Note that there are multiple object and data blocks in a single packet, but currently I can only parse 1 at most.

Bugs Fixed

Ran into a couple bugs, but I’ll just mention 2.

1. When filling the range (part of the object header) I ran into some errors.

   • It turns out that I had forgoten to initilize the space with 0s, and that ment that if less than 8 bytes were assigned, there was garbage memory sitting in the space.
   • Fix was pretty easy, just zeroed out the space and then assigned the data

2. Incorrect DLC data being printed.

   • Another strange bug, not really sure what was causing it
   • Fix was pretty simple, just rewrote the print statements (attached a picture)

Wrap Up

Thanks for enduring that yap fest. Next update will probably be when I add in data parsing, unless I make some major change.Hope you enjoyed it.
Check out the GitHub:

https://github.com/arnabmondal-eg/dnp3

And have a safe, fun, and patriotic Fourth of July!

DNP3 Parser

I genuinely dont know how its been 7 hours. I feel like the majority of my time is fixing bugs and trying to get my code to compile.

Where I’ve Been

It’s been a while since my last devlog, and I have spent the past 7 hours adding a good bit. Main thing is that almost the entire packet is now being parsed, or atleast the hard part.

The entire header, dlc byte, transport header, and application header are parsed, so the only thing left is the Object headers and the data itself.

I also created a new struct to hold all of these functions, so you can just pass in a hex and get all this data out, no need to call each portions functionality anymore.

Whats Next

Moving forward, I have a couple goals/milestones I have to hit before I can ship this project:

1. Parse the rest of the packet
2. Check validity on entire packet (using CRC bytes)
3. Finish Parser UI (almost done, but still some more to be done)
4. Create Control/Main Station UI
5. Create RTU/Secondary Station UI
6. Link backend and frontend using network sockets
7. Create systems for creating packets (I believe this will be much more difficult than parsing them)
8. Simulate “Conversations” between the main and outstations
9. Create binaries for Windows, Mac, and Linux

Just did a little rework to move all header files to the inc/ folder. This should make it less cluttered, but we will see how this holds up.

Next things on the chopping block are:

1. Implement Transport Header Parsing
2. Implement Application Header Parsing
3. Check all CRC bytes in entire packet

Check out the GitHub! https://github.com/arnabmondal-eg/dnp3

Its been a while…

Hello! I have been working on the parser for a while. This post will be an explanation of where I am in the project so far.

I wouldn’t go as far as to say that I am close to done yet, but I’m definitely nearing the halfway point.

The past 2 days I have reworked how packets are interpreted, as well as adding some new helper utilities/functions to make future decoding easier. I also implemented a full bit-by-bit decomposition of the DLC byte in a header!

This was super fun and rewarding as I learned a lot about bit-wise operations in C, as well as strengthening my understand of memory and how it is transferred through a program.

I also learned I can’t spell! Wrote valid as “vaild” over 10 times! I also learned that C doesn’t pass the size of an array automatically! Super fun bugs! \s

Feel free to critique my code on the GitHub! https://github.com/arnabmondal-eg/dnp3