How to remove BOM from UTF-8?
Answer:
# awk '{if(NR==1)sub(/^\xef\xbb\xbf/,"");print}' text.txt
Source: http://stackoverflow.com/questions/1068650/using-awk-to-remove-the-byte-order-mark
Updated: (Suggested by Van Overveldt Peter)
# tail --bytes=+4 text.txt
My preferred command to get rid of the BOM, is:
tail --bytes=+4 UTF8WithBom.txt > UTF8WithoutBom.txt
Thanks for your suggestion, I will include your comments in the post soon.
Thanks again.
This little line of code saved my tail. I had a latex source file that had picked this up and it was refusing to compile it to a PDF because of this nasty little character.
Thanks a bundle guys