CSV logging in Log4J2

Full-featured CSV logging should satisfy following requirements:

1. Dynamic CSV file header
   i.e. any CSV header change triggers automatic file rotation, and a new filename
2. Usage of structured messages
   i.e. rather than a simple String object, message is presented as a Map<String, String>, where every key-value pair maps CSV column name and its value
3. Dynamic mapping between message fields and the CSV header
   i.e. message carries superfluous information and it's up to the runtime configuration (.xml file) to select subset being recorded in the .log file
4. Dynamic routing
   i.e. messages are routed dynamically to its destination file base on the value of one or more fields of the structured message
   

To be honest, there is still some way for Log4J2 to implement fully-featured CSV logging. As of the time of writing:

• #2 is implemented in full
• #3 and #4 are implemented partially 
• #1 has been requested and documented in [2] and [3]

Feature absence, however, is not stopping Product Management from coming up with new requirements, and we will review what can be done today.

Log4J2 has two great features - EventLogger and StructuredDataMessage. Official documentation is available at [1], but in short - it gives us ability to:

• Perform dynamic routing base on the message.type field
• Perform logging in transactional context:
  i.e. we will accumulate messages during the transaction and flush them to the log file only if transaction is considered successful

First, our exemplary EventLogger transactional wrapper looks like:
Secondly, the log4j2.xml file refers to message.type for routing (see the in-line comments for explanation):
Finally, the workaround for CSV header is to store it in another file with the same message.type routing key, and later on use this separate .header file during CSV processing. For instance in example below message.type is the "tableName":

Cheers!


[1] Event logging in Log4J2:
http://logging.apache.org/log4j/2.x/manual/eventlogging.html

[2] RollingFile Appender - callbacks when rolling
https://issues.apache.org/jira/browse/LOG4J2-491

[3] Allow Layout header and footer to be configured in files and to contain properties
https://issues.apache.org/jira/browse/LOG4J2-496