This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
sqoop import-all-tables \ | |
--connect jdbc:mysql://localhost/source_database_name \ (This is the path to access the Source DB using JDBC Driver) | |
--username <username> \ (Source Database Username) | |
--password <password> \ (Source Database password) | |
--database source_database_name \ (The database name will become folder name in target HDFS ) | |
--warehouse-dir <path_name> (This will create folder subdirectories based on importing table names automatically) | |
--autoreset-to-one-mapper (When source tables doesnt have primary key we use this argument) |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
sqoop import \ | |
--connect jdbc:mysql://localhost/source_database_name \ (This is the path to access the Source DB using JDBC Driver) | |
--username \ (Source Database Username) | |
--password \ (Source Database password) | |
--database source_database_name | |
--table source_table_name \ | |
-m 1 | |
--incremental append \ | |
--check-column <columns_name> | |
-last value 1205 |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
sqoop import \ | |
--connect jdbc:mysql://localhost/source_database_name \ (This is the path to access the Source DB using JDBC Driver) | |
--username <username> \ (Source Database Username) | |
--password <password> \ (Source Database password) | |
--database source_database_name \ | |
--where "query" \ | |
--table source_table_name \ | |
--target-dir <path of the file> \ (This is optional and base on requirement) | |
-m 4 (This is based on requirement) |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
sqoop import \ | |
--connect jdbc:mysql://localhost/source_database_name \ (This is the path to access the Source DB using JDBC Driver) | |
--username <username> \ (Source Database Username) | |
--password <password> \ (Source Database password) | |
--database source_database_name \ | |
--table source_table_name \ | |
--taget-dir < path > (we are giving the destination HDFS directory explicitly) | |
-m 1 |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
sqoop import \ | |
--connect jdbc:mysql://localhost/source_database_name \ (This is the path to access the Source DB using JDBC Driver) | |
--username <username> \ (Source Database Username) | |
--password <password> \ (Source Database password) | |
--database source_database_name \ | |
--table source_table_name \ | |
--target-dir <path of the file> | |
-m 4 (using more mappers for parallel increase in perforamce) |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
sqoop import --connect jdbc:mysql://localhost/source_database_name --driver com.mysql.jdbc.Driver --username <username> --password <password> --database source_database_name --table source_table_name -m 1 |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
sqoop export \ | |
--connect jdbc:mysql://localhost/destination_database_name \ | |
--driver com.mysql.jdbc.Driver \ | |
--username <username> \ | |
--password <password> \ | |
--table source_table_name \ (This is the destination table in MySQL) | |
--export-dir /apps/hive/warehouse/movies (This is the Source file to export from HDFS) | |
--input-fields-terminated-by '\0001' (This is the delimiter in HDFS File ) | |
-m 1 |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
sqoop import \ | |
--connect jdbc:mysql://localhost/source_database_name \ (This is the path to access the Source DB using JDBC Driver) | |
--username <username> \ (Source Database Username) | |
--password <password> \ (Source Database password) | |
--database source_database_name \ | |
--table source_table_name \ | |
-m 1 |
NewerOlder