I've read a few non-canonical writings from that time (I'm a Christian). The most interesting orthodox writings are probably the Didache and the Shepherd of Hermas. Among the unorthodox writings, probably the Gospel of Thomas and the Sophia of Jesus Christ. The Sophia of Jesus Christ being particularly, erm, "interesting".