gabidavila

#trans.data_transaction DESC
+----+-------------+---------------+--------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------+--------------------+---------+------------------------------+--------+----------------------------------------------+
| id | select_type | table         | type   | possible_keys                                                                                                                                                     | key                | key_len | ref                          | rows   | Extra                                        |
+----+-------------+---------------+--------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------+--------------------+---------+------------------------------+--------+----------------------------------------------+
|  1 | SIMP

gabidavila

rdohms [17:03] 
am i glad i'm not in brazil now...

rdohms [17:03]
@gabi: how's stuff over there? looks crazy from out here

gabi [17:04] 
nah

gabi [17:05]

gabidavila

Why you need a Data Engineer

Tech evolves pretty quickly, when the buzzword Big Data started showing up more and more the market was in need of people being able to analyse and give meaning for what was collected, for instance, an article from 2012, from Harvard Business Review was entitled: Data Scientist: The Sexiest Job of the 21st Century.

Today we have DBA, Data Scientst, Data Engineer, Data Analyst, a wealth options with the "Data" as prefix. People more often than not put everyone in the same basket and assume everyone knows and have the same set of skills.

From my point of view and perception, and feel free to correct me if wrong, those are the differences:

• DBA - Once the most hated person I ever had in the team. Seriously, why that human doesn't give me the necessary permissions on the database? If I had access, I would have done my job sooner... Well, that was my thought as a Software Enginee

gabidavila

Why you need a Data Engineer

Tech evolves quickly. When the buzzword Big Data started showing up more and more, the market was in need of people able to analyse and give meaning to what was collected. For instance, an article published in 2012 by Harvard Business Review was entitled: Data Scientist: The Sexiest Job of the 21st Century.

Today we have DBA, Data Scientist, Data Engineer, Data Analyst, a wealth of options with the "Data" as prefix. More often than not, people put everyone in the same basket and assume everyone knows and has the same set of skills.

From my point of view and perspective as a Data Engineer, these are the differences:

• DBA - Once the most hated person I ever had in the team. Seriously, why doesn't that human give me the necessary permissions on the database? If I had access, I would have done my job sooner... Well, that was my thought as a Software Engineer at the time. Turns o

gabidavila

-- POSTGRES --

select count(*) from ad;
-- 445000
-- EXPLAIN
-- "Aggregate  (cost=11887.50..11887.51 rows=1 width=0)"
-- "  ->  Seq Scan on ad  (cost=0.00..10775.00 rows=445000 width=0)"


select count(*) from ad  where region_id_fk = 1988;

gabidavila

CREATE TABLE `blog`.`users` (
  `id` INT UNSIGNED NOT NULL AUTO_INCREMENT,
  `name` VARCHAR(45) NOT NULL,
  `email` VARCHAR(255) NOT NULL,
  `created_at` TIMESTAMP NOT NULL DEFAULT CURRENT_TIMESTAMP,
  `updated_at` TIMESTAMP NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
  PRIMARY KEY (`id`));

gabidavila

  /*
    Copy a field part into an output buffer.
    SYNOPSIS
      Field::get_key_image()
      buff   [out] output buffer
      length       output buffer size
      type         itMBR for geometry blobs, otherwise itRAW
    DESCRIPTION
      This function makes a copy of field part of size equal to or
      less than "length" parameter value.

gabidavila

-- Grants access to all databases and tables to the role "dba"
GRANT ALL ON *.* TO 'dba';

-- Grants access to the database "app" to the role "readonly"
GRANT SELECT ON app.* TO 'readonly';

-- Grants access to the database "app" to the role "datawrite"
GRANT INSERT, UPDATE, DELETE ON app.* TO 'app_write';

-- Grants access to developers

gabidavila

CREATE USER 'lisa_simpson'@'%' IDENTIFIED BY 'lisa_pwd';
CREATE USER 'millhouse_houten'@'localhost' IDENTIFIED BY 'millhouse_pwd';
CREATE USER 'homer_simpson'@'localhost' IDENTIFIED BY 'homer_pwd';
CREATE USER 'bart_simpson'@'localhost' IDENTIFIED BY 'bart_pwd';

gabidavila

-- Attributes the users to existing roles
GRANT 'dba' TO 'lisa_simpson'@'%';
GRANT 'developer' TO 'millhouse_houten'@'localhost';
GRANT 'readonly' TO 'homer_simpson'@'localhost', 'bart_simpson'@'localhost';