QuickDataScience | Quick & Easy Data Science

< Previous    Partitioning in Hive We studied the theory part involved in Partitioning in Hive in our previous article. Time to get our hands dirty now.  We will be following the below pattern for the Coding part:-  1. Hadoop Installation . 2. Hive Installation . 3. Static Partitioning.  {The theory part is covered in the previous article.} 4. Dynamic Partitioning. {The theory part is covered in the previous article.} Hope we have installed, and have Hadoop and Hive running. 

 An easy to go guide for installing the Spark in Windows 10. Image taken from Google images 1. Prerequisites Hardware Requirement * RAM — Min. 8GB, if you have SSD in your system then 4GB RAM would also work. * CPU — Min. Quad-core, with at least 1.80GHz JRE 1.8   —   Offline installer for JRE  Java Development Kit — 1.8   A Software for Un-Zipping like   7Zip   or   Win Rar * I will be using 64-bit windows for the process, please check and download the version supported by your system x86 or x64 for all the software. Hadoop * I am using Hadoop-2.9.2, you can also use any other STABLE version for Hadoop.  * If you don’t have Hadoop, you can refer to installing it from   Hadoop: How to install in 5 Steps in Windows 10 . MySQL Query Browser Download Spark Zip * I am using Spark 3.1.1, you can also use any other STABLE version for Spark. * Latest release of Spark is 3.1.2(shown in the image below) released in June'21 Fig 1:- Download Spark-3.1.2

  An easy to go guide for installing SQOOP in Windows 10. Image taken from Google images 1. Prerequisites Hardware Requirement * RAM — Min. 8GB, if you have SSD in your system then 4GB RAM would also work. * CPU — Min. Quad-core, with at least 1.80GHz JRE 1.8   — Offline installer for JRE  Java Development Kit — 1.8   A Software for Un-Zipping like   7Zip   or   Win Rar * I will be using 64-bit windows for the process, please check and download the version supported by your system x86 or x64 for all the software. Hadoop * I am using Hadoop-2.9.2, you can also use any other STABLE version for Hadoop.  * If you don’t have Hadoop, you can refer to installing it from   Hadoop: How to install in 5 Steps in Windows 10 . MySQL Query Browser Download SQOOP zip * I am using SQOOP-1.4.7, you can also use any other STABLE version for SQOOP. Fig 1:- Download Sqoop 1.4.7

  An easy to go guide for installing Hive in Windows 10. Image taken from Google images 1. Prerequisites Hardware Requirement * RAM — Min. 8GB, if you have SSD in your system then 4GB RAM would also work. * CPU — Min. Quad-core, with at least 1.80GHz JRE 1.8  — Offline installer for JRE Java Development Kit — 1.8 A Software for Un-Zipping like  7Zip  or  Win Rar * I will be using 64-bit windows for the process, please check and download the version supported by your system x86 or x64 for all the software. Hadoop * I am using Hadoop-2.9.2, you can also use any other STABLE version for Hadoop. * If you don’t have Hadoop, you can refer to installing it from  Hadoop: How to install in 5 Steps in Windows 10 . MySQL Query Browser Download Hive zip * I am using Hive-3.1.2, you can also use any other STABLE version for Hive. Fig 1:- Download Hive-3.1.2