上篇已经介绍了通过Spark Graphx构建图，在构图过程中要准备一个Long类型的字段作为节点的VertexID，而对于上篇中用到的数据集，显然增加了工作量(将oid[String类型]转换为Long)。
GraphFrames能够避免这样的操作。
GraphFrames
GraphFrames是以DataFrame为基础的构图组件。
(GraphFrames is a package for Apache Spark which provides DataFrame-based Graphs. )
它的目标是保留GraphX功能的同时充分利用Spark DataFrames的优势来扩展GraphX的功能。其扩展功能包括motif 预测，DataFrame序列化以及高性能的图查询。
( It aims to provide both the functionality of GraphX and extended functionality taking advantage of Spark DataFrames. This extended functionality includes motif finding, DataFrame-based serialization, and highly expressive graph queries.)
利用GraphFrames进行构图
主要流程和上篇没太大差异。主要是利用GraphFrames构图时，节点DataFrames需要指明某列为id列(可以使String类型)，边DataFrames需要指明src、dst列(可以使String类型)。代码如下：
//将RDD转为DataFrame
val personsdf = sqlContext.createDataFrame(personsRDD).toDF("id", "name", "no", "groupno", "vertextype")
val groupsdf = sqlContext.createDataFrame(groupsRDD).toDF("id", "name", "tags", "groupno", "vertextype").dropDuplicates("groupno")
val groupsds = groupsdf.select("id", "groupno", "tags")
val relation = personsdf.join(groupsds, personsdf("groupno") === groupsds("groupno")).toDF("src","personname","personno","groupno","personvertextype","dst","groupno","tags").select("src","dst","tags")
val personsds = personsdf.dropDuplicates("id", "no").toDF("id", "name", "tags", "groupno", "vertextype")
val unionds = personsds.union(groupsdf)
val graph = GraphFrame(unionds, relation)
linkurious.js
linkurious.js是纯Javascript开发的js库。它利用Sigma.js进行结构数据的解析及可视化，其提供Canvas、WebGL、SVG三种渲染模式来渲染图的节点和边。
(Linkurious.js is developed in pure Javascript. It uses Sigma.js for its graph data structure and visualization engine, which provides both Canvas, WebGL and SVG renderers for nodes and edges. )
利用linkurious.js进行图的可视化
首先，需要将DataFrames输出成gexf格式的数据结构文件
def graphFrametoGexf(g: GraphFrame) : String = {
"<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n" +
"<gexf xmlns=\"http://www.gephi.org/gexf\" xmlns:viz=\"http://www.gephi.org/gexf/viz\">\n" +
" <graph mode=\"static\">\n" +
" <nodes>\n" +
g.vertices.rdd.map(v => if(v.get(4)=="p")
{" <node id=\"" + v.get(0) + "\" label=\"" +
v.get(1) + "\">\n <viz:color b=\"51\" g=\"51\" r=\"255\"/>\n </node>\n"} else
{" <node id=\"" + v.get(0) + "\" label=\"" +
v.get(1) + "\">\n <viz:color b=\"255\" g=\"51\" r=\"51\"/>\n </node>\n"
}).collect.mkString +
" </nodes>\n" +
" <edges>\n" +
g.edges.rdd.map(e =>
" <edge source=\"" + e.get(0) +
"\" target=\"" + e.get(1) + "\" label=\"" + e.get(2) +
"\" />\n").collect.mkString +
" </edges>\n" +
" </graph>\n" +
"</gexf>"
val strgexf = graphFrametoGexf(graph)
//println(strgexf)
val writer = new PrintWriter(new File("D:\\Tomcat7.0\\webapps\\linkurious.js\\examples\\data\\4444.gexf"))
writer.print(strgexf)
writer.close()
其次，利用linkurious.js进行可视化
sigma.parsers.gexf('data/4444.gexf', {
// container: 'graph-container',
renderer: {
container: document.getElementById('graph-container'),
type: 'canvas'
},
settings: {
edgeColor: 'default',
defaultEdgeColor: '#ccc',
animationsTime: 5000,
drawLabels: true,
labelThreshold: 5,
scalingMode: 'inside',
batchEdgesDrawing: true,
hideEdgesOnMove: true,
sideMargin: 1
}
}, function(s) {
s.graph.nodes().forEach(function (n) {
if (!s.graph.degree(n.id)) {
s.graph.dropNode(n.id);
}
else {
n.x = Math.random();
n.y = Math.random();
}
});
s.refresh();
// Configure the ForceLink algorithm:
var fa = sigma.layouts.configForceLink(s, {
worker: true,
autoStop: true,
background: true,
scaleRatio: 30,
gravity: 3,
easing: 'cubicInOut'
});
// Bind the events:
fa.bind('start stop', function(e) {
console.log(e.type);
document.getElementById('layout-notification').style.visibility = '';
if (e.type == 'start') {
document.getElementById('layout-notification').style.visibility = 'visible';
}
});
// Start the ForceLink algorithm:
sigma.layouts.startForceLink();
});
测试
利用200个用户及400个组进行构图测试，显示效果流畅。(注意：节点名字不能包括Unicode字符，否则linkurious.js无法解析)
