获得RDD[(VertexId, Array[VertexId])]中某个VertexId对应的Array[VertexId]

请问RDD的查询操作用什么?

在Spark中我计算得到一个RDD[(VertexId, Array[VertexId])], 我要获得其中某个VertexId对应的Array[VertexId]大概要怎么写代码啊?

代码:产生一个RDD[(VertexId, Array[VertexId])],并显示其中的具体内容

val neighbors:RDD[(VertexId,Array[VertexId])] = 
social.collectNeighborIds(EdgeDirection.Either) //VertexId是Long的别名
neighbors.collect()

res14: Array[(org.apache.spark.graphx.VertexId, Array[org.apache.spark.graphx.VertexId])] = 
Array((44000000000001232,Array(21000000000002332)), 
(77000000000006009,Array(21000000000002332)), 
(96000000000000101,Array(83000000000004422)), 
(78000000000001982,Array(74000000000000202)), 
(76000000000007234,Array(21000000000002332)), 
(40000000000001381,Array(97000000000008002)), 
(84000000000006934,Array(21000000000002332)), 
(78000000000008000,Array(36000000000009826, 44000000000004587, 
96000000000009876)), (64000000000003255,Array(38000000000001282)), 
(46000000000007691,Array(77000000000008322)), 
(12000000000002345,Array(21000000000002332)), 
(56000000000001000,Array(23000000000002371, 54000000000002771)), 
(34000000000009887,Array(21000000000002332)), 
(98000000000006989,Array(64000000000003021))...

我尝试了lookup()操作,直接报错:key值太大!具体见下面的代码演示:

neighbors.lookup(44000000000001232)
<console>:1: error: integer number too large

直接定义一个变量,赋值44000000000001232,也报错说integer number too large

val id = 78000000000008000
<console>:1: error: integer number too large
       val id = 78000000000008000
阅读 4.2k
1 个回答

Scala、Java中表示Long数的时候后面一定要加L

代码改成:

neighbors.lookup(44000000000001232L)
res9: Seq[Array[org.apache.spark.graphx.VertexId]] = ArrayBuffer(Array(21000000000002332))

val id = 78000000000008000L

![clipboard.png](/img/bVtbVz)
撰写回答
你尚未登录,登录后可以
  • 和开发者交流问题的细节
  • 关注并接收问题和回答的更新提醒
  • 参与内容的编辑和改进,让解决方法与时俱进