Skip to content

Instantly share code, notes, and snippets.

View sebge2emasphere's full-sized avatar

Sébastien Gérard sebge2emasphere

2 followers · 0 following
  • emasphere
View GitHub Profile
@sebge2emasphere
sebge2emasphere / Injection logs
Created January 26, 2018 14:22
[
{
"time": "2018-01-26T14:20:38.079Z",
"entryIndex": 0,
"level": "INFO",
"message": "test",
"details": {
"validationResult": {
"successful": false,
"messages": [
@sebge2emasphere
sebge2emasphere / scratch_139.json
Created January 15, 2018 13:51
{
"id": "718136c7-56c9-48be-ab15-7175bcab846d",
"reference": "analysecom",
"internalReference": "analysecom",
"label": "Analyse Commerciale",
"displayedText": "Analyse Commerciale",
"description": "",
"status": "EDITING",
"entryDefinition": {
"key": {
@sebge2emasphere
sebge2emasphere / region-server-6.7secs.txt
Created October 30, 2017 15:21
2017-10-30 15:09:40,443 DEBUG [RpcServer.listener,port=16020] ipc.RpcServer: RpcServer.listener,port=16020: connection from 172.31.15.200:56894; # active connections: 3
2017-10-30 15:09:40,470 DEBUG [RpcServer.FifoWFPBQ.priority.handler=19,queue=1,port=16020] hdfs.DFSClient: Connecting to datanode 172.31.15.200:50010
2017-10-30 15:09:40,470 TRACE [RpcServer.FifoWFPBQ.priority.handler=19,queue=1,port=16020] hdfs.BlockReaderFactory: BlockReaderFactory(fileName=/user/hbase/data/hbase/meta/1588230740/info/70870e4a202a4ac681d63edc151488e7, block=BP-983401122-172.31.15.200-1508568126354:blk_1073746500_5714): trying to create a remote block reader from a TCP socket
2017-10-30 15:09:40,471 TRACE [RpcServer.FifoWFPBQ.priority.handler=19,queue=1,port=16020] hdfs.BlockReaderFactory: nextTcpPeer: reusing existing peer NioInetPeer(Socket[addr=/172.31.15.200,port=50010,localport=51498])
2017-10-30 15:09:40,471 TRACE [RpcServer.FifoWFPBQ.priority.handler=19,queue=1,port=16020] datatransfer.DataTransferProtocol: Sending Data
@sebge2emasphere
sebge2emasphere / block-20000
Created October 29, 2017 09:13
2017-10-26 15:30:22 TRACE AbstractRpcClient:236 - Call: Scan, callTime: 5ms
2017-10-26 15:30:22 TRACE ScannerCallableWithReplicas:234 - Setting current scanner as id=-1 associated with replica=0
2017-10-26 15:30:22 TRACE MetaCache:175 - Merged cached locations: [region=flow,,1508569108231.4e4d11e77742818f3c7ee4782f787b16., hostname=ip-172-31-15-200.ec2.internal,16020,1509031400543, seqNum=2772]
2017-10-26 15:30:22 TRACE AbstractRpcClient:236 - Call: Scan, callTime: 1ms
2017-10-26 15:30:22 TRACE ScannerCallableWithReplicas:234 - Setting current scanner as id=23 associated with replica=0
2017-10-26 15:30:22 TRACE AbstractRpcClient:236 - Call: Scan, callTime: 501ms
2017-10-26 15:30:22 TRACE ScannerCallableWithReplicas:234 - Setting current scanner as id=23 associated with replica=0
2017-10-26 15:30:22 TRACE ClientScanner:651 - number results from RPC: 20000,partial != null: false,number of partials so far: 0
2017-10-26 15:30:23 TRACE AbstractRpcClient:236 - Call: Scan, callTime: 582ms
2017-10-26 15:30:23 TRACE S
@sebge2emasphere
sebge2emasphere / spark-8seconds-jb
Created October 26, 2017 14:42
2017-10-26 14:35:57 TRACE MessageDecoder:47 - Received message OneWayMessage: OneWayMessage{body=NettyManagedBuffer{buf=CompositeByteBuf(ridx: 5, widx: 8470, cap: 8470, components=9)}}
2017-10-26 14:35:57 INFO CoarseGrainedExecutorBackend:54 - Got assigned task 62
2017-10-26 14:35:57 INFO Executor:54 - Running task 0.0 in stage 72.0 (TID 62)
2017-10-26 14:35:57 DEBUG Executor:58 - Task 62's epoch is 40
2017-10-26 14:35:57 DEBUG BlockManager:58 - Getting local block broadcast_65
2017-10-26 14:35:57 TRACE BlockInfoManager:62 - Task 62 trying to acquire read lock for broadcast_65
2017-10-26 14:35:57 DEBUG BlockManager:58 - Block broadcast_65 was not found
2017-10-26 14:35:57 INFO TorrentBroadcast:54 - Started reading broadcast variable 65
2017-10-26 14:35:57 DEBUG TorrentBroadcast:58 - Reading piece broadcast_65_piece0 of broadcast_65
2017-10-26 14:35:57 DEBUG BlockManager:58 - Getting local block broadcast_65_piece0 as bytes
@sebge2emasphere
sebge2emasphere / Executor debug
Last active October 18, 2017 08:29
2017-10-18 07:30:47 INFO CoarseGrainedExecutorBackend:54 - Got assigned task 20
2017-10-18 07:30:47 INFO CoarseGrainedExecutorBackend:54 - Got assigned task 21
2017-10-18 07:30:47 INFO Executor:54 - Running task 0.0 in stage 18.0 (TID 20)
2017-10-18 07:30:47 DEBUG Executor:58 - Task 20's epoch is 10
2017-10-18 07:30:47 DEBUG BlockManager:58 - Getting local block broadcast_18
2017-10-18 07:30:47 DEBUG BlockManager:58 - Block broadcast_18 was not found
2017-10-18 07:30:47 INFO TorrentBroadcast:54 - Started reading broadcast variable 18
2017-10-18 07:30:47 DEBUG TorrentBroadcast:58 - Reading piece broadcast_18_piece0 of broadcast_18
2017-10-18 07:30:47 DEBUG BlockManager:58 - Getting local block broadcast_18_piece0 as bytes
2017-10-18 07:30:47 DEBUG BlockManager:58 - Getting remote block broadcast_18_piece0
@sebge2emasphere
sebge2emasphere / gist:0fbac847fb7141b22bced4114792a116
Created October 17, 2017 16:28
Hadoop Spark
JavaPairRDD
.fromRDD(
context.newAPIHadoopRDD(
initializeConfiguration(executorContext, context),
TableInputFormat.class,
ImmutableBytesWritable.class,
Result.class
),
ClassTag$.MODULE$.apply(ImmutableBytesWritable.class),
ClassTag$.MODULE$.apply(Result.class)