Normalize newlines

3 types of newlines

There are 3 ways to represent a newline.

Unix: using single character LF, which is byte 10 (0x0a), represented as “” in Go string literal.

Windows: using 2 characters: CR LF, which is bytes 13 10 (0x0d, 0x0a), represented as “” in Go string literal.

Mac OS: using 1 character CR (byte 13 (0x0d)), represented as “” in Go string literal. This is the least popular.

When splitting strings into lines you have to decide how you’ll handle this.

You can assume that your code will only see e.g. Unix style line endings and only handle “”, but this won’t work at all on files with Mac line endings and files with Windows line endings will have a CR character in them.

A simple way to handle multiple newline representations is to normalize the newlines and then operate on the normalized version.

Finally you can write code that handles all newline endings. Inevitably, such code is a bit more complicated.

Normalize newlines

// NormalizeNewlines normalizes \r\n (windows) and \r (mac)
// into \n (unix)
func NormalizeNewlines(d []byte) []byte {
	// replace CR LF \r\n (windows) with LF \n (unix)
	d = bytes.Replace(d, []byte{13, 10}, []byte{10}, -1)
	// replace CF \r (mac) with LF \n (unix)
	d = bytes.Replace(d, []byte{13}, []byte{10}, -1)
	return d
}

"new\nline"