hadoop - Extracting Column name from Twitter JSON File -


i trying analyse twitter data using hadoop. have created hive table according tweet had previously. have again downloaded twitter data , problem new columns came in tweet not present in previous tweet data. question is there way can find maximum number of columns tweet can create hive table it. helpless far kindly

if have tweets in json format make table in hive using below query

create external table tweets (    id bigint,    created_at string,    source string,    favorited boolean,    retweet_count int,    retweeted_status struct<       text:string,       user:struct<screen_name:string,name:string>>,    entities struct<       urls:array<struct<expanded_url:string>>,       user_mentions:array<struct<screen_name:string,name:string>>,       hashtags:array<struct<text:string>>>,    text string,    user struct<       screen_name:string,       name:string,       friends_count:int,       followers_count:int,       statuses_count:int,       verified:boolean,       utc_offset:int,       time_zone:string>,    in_reply_to_screen_name string )  row format serde 'com.cloudera.hive.serde.jsonserde' location '/user/hive/warehouse/tweets';

download jar form http://files.cloudera.com/samples/hive-serdes-1.0-snapshot.jar , add

add jar /home/kishore/hive-0.9.0/lib/hive-serdes-1.0-snapshot.jar;

-

Comments

Post a Comment

Popular posts from this blog

javascript - Trigger mouseenter when an animated element touches mouse -

i'm new web development , i'm building game in object avoid touching animated circles pointer. set mouseenter event (full jsfiddle below): $("#"+circlename).mouseenter(function(){ $(this).attr("class", "redcircle"); }); this turns circles red when user moves pointer them. however, when mouse stationary, circles can pass through safely. there better way this? by changing mouseenter event hover event, event triggered when animated circle touches stationary mouse. however, there seems delay. know why might be? this sounds it's related issue found here, means may have related browser bug: getting mouseenter event still mouse entering animated element fiddle: http://jsfiddle.net/nbsteuv/6yok3s56/4/ i store last mouse position , whenever circle moves, check whether has intersected mouse position. var mousex, mousey; $( "#container-element-id" ).mousemove(function( event ) { mousex = event.pagex; mouse...
Read more

json - Zend error Connection -

i have problem code, first code <?php require_once('zend/json.php'); require_once('zend/db.php'); //require_once 'zend/db/adapter/pdo/pgsql.php'; class jsonvi { protected $_db; public function _koneksi () { try { $this->_db = zend_db::factory('pdo_pgsql',array( 'host'=>'localhost', 'username'=>'stet', 'password'=>'test', 'dbname'=>'test' )); return $this->_db; } catch(zend_db_exception $e) { return $e->getmessage(); } } public function getdata() { $db = $this->_koneksi(); try { $sql = "select * info "; ...
Read more

java - Using Spring @Transactional with a combination of readOnly and write, when does this entity get committed? -

given class callingclass { @autowired someservice someservice; doit() { someservice.readonlycall(); } } @transactional(readonly = true) class someservice { @autowired anotherservice anotherservice; readonlycall() { // logic may do... anotherservice.writecall(); } } class anotherservice { @transactional // not readonly writecall() { // save entity } } will entity committed database when writecall method exits or when readonlycall method exits? and how can modify commits after writecall method? the entity not committed database in both method exits. because your someservice class has read-only (which have specify). set flushmode.never prevent commit transaction your someservice class , anotherservice class has required propagation type(which default , have not specify other type both class). create new transaction if doesn't exist use current transaction if there is. in case someservice class begin transaction read-only , an...
Read more