Graph neural networks (GNNs) have attracted an increasing attention in recent years. However, most existing state-of-the-art graph learning methods only focus on node features and largely ignore the edge features that contain rich information about graphs in modern applications. In this paper, we propose a novel model to incorporate Node and Edge features in graph Neural Networks (NENN) based on a hierarchical dual-level attention mechanism. NENN consists of node-level attention layer and edge-level attention layer. The two types of layers of NENN are alternately stacked to learn and aggregate embeddings for nodes and edges. Specifically, the node-level attention layer aims to learn the importance of the node based neighbors and edge based neighbors for each node, while the edge-level attention layer is able to learn the importance of the node based neighbors and edge based neighbors for each edge. Leveraging the proposed NENN, the node and edge embeddings can be mutually reinforced. Extensive experiments on academic citation and molecular networks have verified the effectiveness of our proposed graph embedding model.