README.md

590V assignment 2

1. A scatterplot is a plot that uses cartesian coordinates to display values of a dataset. It is a graph of points which helps us to visualize the relationship between different dimensions(features).
   • The basic algorithm for creating a scatter plot is follows:

   #Reference: Interactive Data Visualization: Foundations, Techniques, and Applications, Second Edition by Daniel A. Keim, Georges G. Grinstein, and Matthew O. Ward
   
   def scatterplot(xDim,yDim,cDim,rDim,rMin,rMax):
       for each record i:
           x = NORMALIZE(i,xDim)
           y = NORMALIZE(i,yDim)
           r = NORMALIZE(i,rDim,rMin,rMax)
           MAPCOLOR(i,cDim)#Maps the value of the dimension of the data point to a color
           DRAW_CIRCLE(x,y,r) 
   

   • We can gain several insights from the data using a scatter plot. We can see the correlation between the different dimensions(features) of the data point. We can formulate several hypothesis from the data, eg: in our data the MPG decreases as horsepower increases. We can also visualize and fit a curve through the data points. Scatterplots also helps us to visualize multiple dimensions in one go using different features like size, colour, shape etc. Several interesting discoveries can also be made by using a scatter plot. We can also use scatter plots to prove and convince some points by showing the patterns in data.
   • Following are some features of the scatter plot that can be controlled:
     • color of the point: This can be used to show categorical features in general. We can assign a color to a category. This can be controlled by choosing any feature and assigning a color to a value.
     • radius/size of the data point: This can be used to show an extra dimension. We can show continuous as well as some categorical data like the number of cylinders. This can be controlled by normalizing the dimension values and mapping it to radius range.
     • shape of the data point: We can use different shapes for different categories of the data.
     • legends: Legends can give additional info about some dimension by providing a quick look up map. They can be derived from the dimension values of a data.
     • Opacity/transparency of a data point: Varying degree of opacity can show different values of the data point.
     • Tooltip/additional info on mouse over: Additional information of the dimension can be provided when mouse is hovered over a point.
     • Drop downs for changing x and y dimension: x and y axis values can be made selectable by providing drop downs.
     • 3D scatter plot: We can add an extra dimension and visualize the scatterplot in 3D space. This gives us one more dimension.
     • Regression/Best fit line: A regression/best fit line can be used to best represent/fit the data. We can use any linear regression technique and regularization for this.
     • Filters: Filters can be used to filter data on a dimension. It can be a check box with values for a dimension or we can provide fine grain controls like greater than/less than.
     • Tick marks can be used to show the values at intermediate points. If we make tick marks go all across the graph it would help the user to identify data points.
     • Highlighting data points on mouse over: We can highlight data points on mouse over to make the data point explicitly stand out in a scatter plot.
     • X and Y axis labels: X and Y axis labels should be provided to understand the dimensions being represented.
     • Selector brushes: We can use a cricular or lasso selector to select a subset of points to compute some metric.
     • Reset button: A reset button can be provided to reset values to default.
     • Segregated clusters: Clusters can be identified with well specified boundaries. A circle can be drawn around clusters.
     • Derived dimensions: Derived dimensions which is derived from based dimensions can be used to represent data. Like velocity= distance/time. We can allow the user to define these derived dimensions.
     • Zoom and pan in a scatter plot: We can enable the users to zoom and pan the data points, thus enabling them to do more fine grained analysis on the data.
2. A scatterplot at its basic form is a 2D or 3D representation of data. Most of the problems we deal in real life has a lot of dimensions. I am suggesting a scatter plot where we use PCA dimensionality reduction to reduce the scatter plot dimensions and project the data into a 2D or 3D scatter plot. Here we can more easily identify clusters of data. On mouse over we should be able to see the original data point with its constituent dimensions.
3. I obtained the data from Prof. Curran's video: https://github.com/curran/d3-in-motion/blob/master/data/cars.csv. The data contains car data with the following features: mpg,cylinders,displacement,horsepower,weight,acceleration,year,origin,name. It has 392 data points.
4. Please refer the scatter plot.
5. 2 other variables: Color and Radius(size) of the circle can be changed.
6. Mouse over/Tool tip is enabled.
7. An interesting trend to visualize is the relationship between miles per gallon and horsepower for the data set. We can see a clear decline in miles per gallon as we increase the horsepower. This suggests that we will have to make a trade off between power and economy of the vehicle while buying one. Datsun 280-zx is an outlier which gives a very good mpg of 32.7 for 132 horsepower. It is also interesting to see that most of the high power vehicles are from USA and all the high power vehicles have 8 cylinders which suggest a V8 engine as expected. I believe that this visualization of the trend between MPG and horsepower is best captured in a scatter plot. This is because scatter plot is one of the best visualization for discovering the relationship between variables. A line graph using the best fit curve to the data is another visualization that can be used.

References:

1. https://www.manning.com/livevideo/d3-js-in-motion
2. Interactive Data Visualization: Foundations, Techniques, and Applications, Second Edition by Daniel A. Keim, Georges G. Grinstein, and Matthew O. Ward
3. Lab 3 590V Files

cars.csv

index.html

<!DOCTYPE html>
<html>
  <head>
    <link rel="stylesheet" href="https://maxcdn.bootstrapcdn.com/bootstrap/4.0.0/css/bootstrap.min.css" integrity="sha384-Gn5384xqQ1aoWXA+058RXPxPg6fy4IWvTNh0E263XmFcJlSAwiGgFAW/dAiS6JXm" crossorigin="anonymous">
    <meta charset="utf-8">
    <meta name="viewport" content="width=device-width">
    <script src="https://d3js.org/d3.v4.min.js"></script>
    <script src="https://cdnjs.cloudflare.com/ajax/libs/d3-legend/2.24.0/d3-legend.min.js"></script>
    <script
  src="http://code.jquery.com/jquery-3.3.1.min.js"
  integrity="sha256-FgpCb/KJQlLNfOu91ta32o/NMZxltwRo8QtmkMRdAu8="
  crossorigin="anonymous"></script>
    <title>Scatter plot</title>
  <link rel='stylesheet' href='style.css'>
  </head>
  <body>
    <div id='vis-container'>
    <div class='yaxis'>
    Y axis dimension: &nbsp;&nbsp;&nbsp;&nbsp;X axis dimension&nbsp;&nbsp;&nbsp;&nbsp;Shape dimension:&nbsp;&nbsp;&nbsp;&nbsp;Color dimension:<br/><span class='yvalue'></span>&nbsp;&nbsp;&nbsp;&nbsp;<span class='xvalue'></span>&nbsp;&nbsp;&nbsp;&nbsp;
    <span class='shapevalue'></span>&nbsp;&nbsp;&nbsp;&nbsp;<span class='colorvalue'></span>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<button type="button" class="btn btn-outline-primary" id="resetcolor">Reset Color</button>
    </div>
    <svg width='960' height='500'></svg>
  </div>
    <script src="script.js">
    </script>
  </body>
</html>

script.js

//Configuration of x,y, shape and color. 
var selectData = [{ "text": "Horsepower" }, { "text": "Displacement" },
{ "text": "MPG" }, { "text": "Weight" },
{ "text": "Acceleration" },
];

var xselectData = [{ "text": "Weight" }, { "text": "Horsepower" }, { "text": "Displacement" },
{ "text": "MPG" },
{ "text": "Acceleration" },
];

var shapeselectData = [{ "text": "Cylinders" }, { "text": "Weight" }, { "text": "Horsepower" }, { "text": "Displacement" },
{ "text": "MPG" },
{ "text": "Acceleration" },
];

var colorselectData = [{ "text": "origin" }, { "text": "Cylinders" }]

const svg = d3.select('svg')

const xLabel = 'Weight';
const yLabel = 'Horsepower';
const colorLabel = 'Origin';

const height = svg.attr('height');
const width = svg.attr('width');

const shapeDefaultValue = d => d.cylinders;
const xValue = d => d.weight;
const yvalue = d => d.horsepower;
const colorValue = d => d.origin;

//Domain and Range. domain data, range pixel coordinates
const shapeScale = d3.scaleLinear();
const xScale = d3.scaleLinear();
const yScale = d3.scaleLinear();
const colorScale = d3.scaleOrdinal()
    .range(d3.schemeCategory10);

//Setting margins for cut off data

const margin = {
    left: 100,
    right: 300,
    top: 20,
    bottom: 100
};

//Spread across inner rectangles within margins

const innerWidth = width - margin.left - margin.right;
const innerHeight = height - margin.top - margin.bottom;

const g = svg.append('g').attr('transform', `translate(${margin.left},${margin.top})`)

//Putting in the axis
const xAxisG = g.append('g').attr('transform', `translate(0,${innerHeight})`)
const yAxisG = g.append('g')
const colorLegendG = g.append('g').attr('transform', `translate(${innerWidth + 60},150)`);

//Axis Labels

xAxisG.append('text').attr('class', 'xaxis-label').text(xLabel).attr('x', innerWidth / 2)
    .attr('y', 68)

yAxisG.attr('class', 'yAxis').append('text').attr('class', 'yaxis-label')
    .attr('transform', `rotate(-90)`)
    .attr('x', -innerHeight / 2)
    .attr('y', -55)
    .style('text-anchor', 'middle')
    .text(yLabel)

colorLegendG.append('text').attr('class', 'caxis-label')
    .attr('y', -40)
    .attr('x', -10)
    .text(colorLabel);

//Drawing axis and color legend
const xAxis = d3.axisBottom().scale(xScale).tickPadding(15).tickSize(-innerHeight);
const yAxis = d3.axisLeft().scale(yScale).ticks(5).tickPadding(15).tickSize(-innerWidth);
var colorLegend = d3
    .legendColor()
    .scale(colorScale)
    .shape('circle')
    .shapeRadius(6)
    .shapePadding(2)
    .on("cellclick", function (type) {
        d3.selectAll(".cell")
            .style("opacity", 0.1);
        d3.select(this)
            .style("opacity", 1);
        d3.selectAll(".data-circle")
            .style("opacity", 0)
            .filter(function (d) {
                return d["origin"] == type;
            })
            .style("opacity", 1)
    });

const row = d => {
    d.horsepower = +d.horsepower;
    d.weight = +d.weight;
    d.mpg = +d.mpg;
    d.acceleration = +d.acceleration;
    d.displacement = +d.displacement;
    d.cylinders = +d.cylinders;
    return d;
};


//Tool tip for mouse over
var tooltip = d3.select('#vis-container')
    .append('div').attr('class', 'tooltip')
    .style('opacity', 0)


//Tooltip handler
var tipMouseover = function (d) {

    var chtml = ` <b>Name</b>: ${d.name}<br/>
    <b>Weight</b>: ${d.weight}<br/>
          <b>Origin</b>: ${d.origin}<br/>
          <b>Cylinders</b>: ${d.cylinders}<br/>
          <b>MPG</b>: ${d.mpg}<br/>
          <b>Displacement</b>: ${d.displacement}<br/>
          <b>Acceleration</b>: ${d.acceleration}<br/>
          <b>Horsepower</b>: ${d.horsepower}<br/>`;

    tooltip.html(chtml)
        .style("left", (d3.event.pageX + 15) + "px")
        .style("top", (d3.event.pageY - 28) + "px")
        .transition()
        .duration(200)
        .style("opacity", .9)

};
// tooltip mouseout event handler
var tipMouseout = function (d) {
    tooltip.transition()
        .duration(300) // ms
        .style("opacity", 0);
};


//Reset color handler
$('#resetcolor').click(function () {
    resetColors();
});

//Utility function for adding a selector
function addSelector(selector, change, data) {
    var yinput = d3.select(selector)
        .append('select')
        .attr('id', 'xselect').on('change', change)
        .selectAll('option')
        .data(data)
        .enter().append('option')
        .attr('value', function (d) { return d.text })
        .text(function (d) { return d.text; })
}
//Utility function for resetting colors
function resetColors() {
    d3.selectAll(".data-circle").style('opacity', 1);
    d3.selectAll(".cell").style('opacity', 1);
}


//Read CSV and execute call back
d3.csv('cars.csv', row, data => {

    //Setting scales
    shapeScale.domain(d3.extent(data, shapeDefaultValue))
        .range([2, 7]).nice();

    xScale.domain(d3.extent(data, xValue))
        .range([0, innerWidth]).nice();

    yScale.domain(d3.extent(data, yvalue))
        .range([innerHeight, 0]).nice();

    colorScale.domain(data.map(colorValue))

    //Draw the circles of the scatter plot
    g.selectAll('circle')
        .data(data).enter().append('circle')
        .attr('class', 'data-circle')
        .attr('cx', d => xScale(xValue(d)))
        .attr('cy', d => yScale(yvalue(d)))
        .attr('r', d => shapeScale(shapeDefaultValue(d)))
        .attr('fill', d => colorScale(colorValue(d)))
        .on('mouseover', tipMouseover)
        .on('mouseout', tipMouseout);

    xAxisG.call(xAxis);
    yAxisG.call(yAxis);
    colorLegendG.call(colorLegend)
        .selectAll('.cell text')
        .attr('dy', '0em');

    //Adding the y axis selector
    addSelector('.yvalue', yChange, selectData);

    //Adding x axis selector
    addSelector('.xvalue', xChange, xselectData);

    //Adding shape axis selector

    addSelector('.shapevalue', shapeChange, shapeselectData);

    //Adding color axis selector

    addSelector('.colorvalue', colorChange, colorselectData);

    //Function to populate new y values and y axis values for changes in dimension
    function yChange() {
        var value = this.value.toLowerCase();
        yScale.domain(d3.extent(data, function (d) { return d[value] }))
            .range([innerHeight, 0]).nice();
        yAxis.scale(yScale);
        yAxisG.call(yAxis);
        d3.select('.yaxis-label')
            .text(value);
        d3.selectAll('.data-circle')
            .attr('cy', function (d) {
                // console.log(d)
                return yScale(d[value])
            });
    }

    //Function to populate new x values and x axis values for changes in dimension
    function xChange() {
        var value = this.value.toLowerCase();
        xScale.domain(d3.extent(data, function (d) { return d[value] }))
            .range([0, innerWidth]).nice();
        xAxis.scale(xScale);
        xAxisG.call(xAxis);
        d3.select('.xaxis-label')
            .text(value);
        d3.selectAll('.data-circle')
            .attr('cx', function (d) {
                // console.log(d)
                return xScale(d[value])
            });
    }

    //Function to populate new values for changes in shape
    function shapeChange() {
        var value = this.value.toLowerCase();
        shapeScale.domain(d3.extent(data, function (d) { return d[value] }))
            .range([2, 7]).nice();
        d3.selectAll('.data-circle')
            .attr('r', function (d) {
                // console.log(d)
                return shapeScale(d[value])
            });
    }
    //Function to populate new color legend
    function colorChange() {
        resetColors();
        var value = this.value.toLowerCase();
        d3.select('.caxis-label')
            .text(value);
        colorScale.domain(data.map(d => d[value]));
        // colorScale.domain(data.map(colorValue));
        d3.selectAll('.data-circle').attr('fill', function (d) {
            return colorScale(d[value]);
        });
        d3.selectAll('.legendCells').remove();
        var newColorLegend = d3
            .legendColor()
            .scale(colorScale)
            .shape('circle')
            .shapeRadius(6)
            .shapePadding(2)
            .on("cellclick", function (type) {
                d3.selectAll(".cell")
                    .style("opacity", 0.1);
                d3.select(this)
                    .style("opacity", 1);
                d3.selectAll(".data-circle")
                    .style("opacity", 0)
                    .filter(function (d) {
                        // console.log(value);
                        return d[value] == type;
                    })
                    .style("opacity", 1)
            });
        colorLegendG.call(newColorLegend)
            .selectAll('.cell text')
            .attr('dy', '0em');
    }
});

style.css

.tick .leged-cells{
  font-size: 2em;
}

.tick text{
  font-size: 2em
}
.axis-label, .xaxis-label,.yaxis-label,.caxis-label{
  font-size: 30pt;
  fill: black;
  font-family: sans-serif;
}

body{
  margin-left:20px;
  margin-right:20px;  
}

.tooltip {
  position: absolute;
  font-size: 16px;
  width:  auto;
  height: auto;
  pointer-events: none;
  background-color: white;
}

.yaxis{
  position: relative;
  left: 92px;
  padding-bottom: 10px
}