Grid should not display automatically unless you have done something like save the document as a image file with grids turned on. If you save a document with visible grids and guides Photoshop will save the information like it save other meta-data like Print DPI resolution raw settings etc. Photoshop will use this information when you open the image file. You could also turn grids on with an open event handler.
If it come up with new documents turn off the grid and save a document without the grid. Sometime or for somethings Photoshop also remembers what you used the last save like rulers on/off grid on/off last folder used. I have to admit I do get confuse as to what Photoshop does at times. Like Ruler On/Off Grit On/Off seems to be relates to the last save when I know Guide lines and RAW setting get save with the file.