Microsoft Interview Question SDE-2s
Country: India
Interview Type: In-Person
Write To File --- i answered the same with Serialize with a lock and after the interviewer told about Performance, i told about the Protecting of just the Write Offset. And told about the Live and Frozen buffer.
The per CPU log idea is a good one... Need to think about it...
When a hint was given about the Not all logs are written in File --- After some research i found the below (Chain Of Responsibility Pattern)
class Logger {
static int ERR = 1;
static int INFO = 2;
static int DEBUG = 3;
virtual void message(String msg, int priority) {
writeMessage(msg);
next.message(msg, priority);
}
} ;
class StdoutLogger : Logger {
void writeMessage(String msg) {
...
}
};
class EmailLogger : Logger {
void writeMessage(String msg) {
...
}
} ;
class StderrLogger : Logger {
void writeMessage(String msg) {
...
}
} ;
@Ramesh N: I believe the
class Logger
is a C++ code, then its
message
function has to be virtual.
how about using already existing infrastructures like log4j in java and windows Trace in .NET
The Interviewer wanted me to Design the Logging mechanism as if it was design to be part of OS. Not using any existing facilities.
@The Artist --- Anyway thanks for the input, i am going through the log4j implementation,to understand how it is designed.
If it has to be thread safe and provide high performance, then the first thing to do is to decouple the serialization into the file with the rest of the logic, since file I/O is much slower than memory operations.This knowledge inherently yields a dedicated thread for the logger to serialize its FIFO cache of messages, which can be implemented with the command pattern, as mentioned by the poster. Anyway, access to the shared resources has to be protected with some type of OS-specific locks or similar mechanisms. On Windows platforms, you can use the QueueUserAPC, which will shift the locking from the user-space to the OS's kernel, while on Linux platforms, you will eventually come to mutices and the like.
Writing to a file is a two step operation
1) write data in file
2) increment file descriptor pointer by buffer size
File opened in append mode make sure atomicity of above both operation. OS take care of performing the above both operation in thread safe fashion.
Solution for thread safe is open log file in APPEND mode.
Write to a file
- smallchallenges January 07, 2014+ Most file systems serialize writes to a file. But, we cannot predict the order in which writes end up in a file. So, if you need absolute incoming order to be maintained, then serialize with a lock. This leads to a performance issue due to lock contention.
+ We can work around it by using something like mmap() and just protecting the write offset into the file and taking the actual write out of the performance path i.e. the lock will protect only the mmap offset movement. But, if we assume that the number of bytes logged is less, then we do reduce contention by much.
Write to a per-cpu in-mem buffer and transfer to a file offline.
+ Each cpu can log stuff into its own buffer i.e. no lock contention issues.
+ But, if you need ordering, you have to add a timestamp to each entry that can be sorted offline. This assumes that post processing of the log entries is an allowed relaxation of rules.
+ Still, when the in-core buffer is full, contents need to be written to the disk..so, will logging stall at that time? The usual solution for this is to have two buffers: live and frozen. While the frozen buffer is being written to disk, the live one continues to take the logging entries.