diff --git a/03_homework/homework_1.md b/03_homework/homework_1.md index c50fe783c..6bba5d5d9 100644 --- a/03_homework/homework_1.md +++ b/03_homework/homework_1.md @@ -1,4 +1,5 @@ # Homework 1: farmersmarket.db +homework_1_photo - Due on Thursday, May 16 at 11:59pm - Weight: 8% of total grade @@ -76,3 +77,6 @@ Please do not pick the exact same tables that I have already diagramed. For exam - ![01_farmers_market_conceptual_model.png](./images/01_farmers_market_conceptual_model.png) - The column names can be found in a few spots (DB Schema window in the bottom right, the Database Structure tab in the main window by expanding each table entry, at the top of the Browse Data tab in the main window) +homework_1_photo + + diff --git a/03_homework/homework_2.sql b/03_homework/homework_2.sql index f0788ee2a..4c878a9c8 100644 --- a/03_homework/homework_2.sql +++ b/03_homework/homework_2.sql @@ -1,29 +1,50 @@ --SELECT /* 1. Write a query that returns everything in the customer table. */ - +Select * +from customer /* 2. Write a query that displays all of the columns and 10 rows from the cus- tomer table, sorted by customer_last_name, then customer_first_ name. */ - - +Select * +from customer +order by customer_last_name, customer_first_name +LIMIT 10 --WHERE /* 1. Write a query that returns all customer purchases of product IDs 4 and 9. */ -- option 1 +SELECT * +from customer_purchases +where product_id = 4 + or product_id = 9 + -- option 2 +SELECT * +FROM customer_purchases +WHERE product_id IN (4, 9) + /*2. Write a query that returns all customer purchases and a new calculated column 'price' (quantity * cost_to_customer_per_qty), filtered by vendor IDs between 8 and 10 (inclusive) using either: 1. two conditions using AND 2. one condition using BETWEEN */ -- option 1 - + +SELECT *, + quantity * cost_to_customer_per_qty AS price +FROM customer_purchases +WHERE vendor_id >= 8 AND vendor_id <= 10 -- option 2 + +SELECT *, + quantity * cost_to_customer_per_qty AS price +FROM customer_purchases +WHERE vendor_id BETWEEN 8 AND 10 --CASE /* 1. Products can be sold by the individual unit or by bulk measures like lbs. or oz. @@ -31,12 +52,35 @@ Using the product table, write a query that outputs the product_id and product_n columns and add a column called prod_qty_type_condensed that displays the word “unit” if the product_qty_type is “unit,” and otherwise displays the word “bulk.” */ +SELECT product_id, product_name, + CASE + WHEN product_qty_type = 'unit' THEN 'unit' + ELSE 'bulk' + END AS prod_qty_type_condensed +FROM product /* 2. We want to flag all of the different types of pepper products that are sold at the market. add a column to the previous query called pepper_flag that outputs a 1 if the product_name contains the word “pepper” (regardless of capitalization), and otherwise outputs 0. */ +SELECT product_id, product_name, + CASE + WHEN product_qty_type = 'unit' THEN 'unit' + ELSE 'bulk' + END AS prod_qty_type_condensed, + CASE + WHEN LOWER(product_name) LIKE '%pepper%' THEN 1 + ELSE 0 + END AS pepper_flag +FROM product + --JOIN /* 1. Write a query that INNER JOINs the vendor table to the vendor_booth_assignments table on the vendor_id field they both have in common, and sorts the result by vendor_name, then market_date. */ + +SELECT vendor.vendor_id,vendor.vendor_name, vendor_booth_assignments.market_date +FROM vendor +INNER JOIN vendor_booth_assignments ON vendor.vendor_id = vendor_booth_assignments.vendor_id +ORDER BY vendor.vendor_name, vendor_booth_assignments.market_date + diff --git a/03_homework/homework_3.sql b/03_homework/homework_3.sql index 99f489f31..7c5f80068 100644 --- a/03_homework/homework_3.sql +++ b/03_homework/homework_3.sql @@ -2,7 +2,10 @@ /* 1. Write a query that determines how many times each vendor has rented a booth at the farmer’s market by counting the vendor booth assignments per vendor_id. */ - +SELECT vendor_id, + COUNT(*) AS booth_rent_count +FROM vendor_booth_assignments +GROUP BY vendor_id /* 2. The Farmer’s Market Customer Appreciation Committee wants to give a bumper sticker to everyone who has ever spent more than $2000 at the market. Write a query that generates a list @@ -10,7 +13,12 @@ of customers for them to give stickers to, sorted by last name, then first name. HINT: This query requires you to join two tables, use an aggregate function, and use the HAVING keyword. */ - +SELECT c.customer_id, c.customer_first_name,c.customer_last_name +FROM customer c +JOIN customer_purchases cp ON c.customer_id = cp.customer_id +GROUP BY c.customer_id, c.customer_first_name, c.customer_last_name +HAVING SUM(cp.quantity * cp.cost_to_customer_per_qty) > 2000 +ORDER BY c.customer_last_name, c.customer_first_name --Temp Table /* 1. Insert the original vendor table into a temp.new_vendor and then add a 10th vendor: @@ -23,8 +31,11 @@ When inserting the new vendor, you need to appropriately align the columns to be -> To insert the new row use VALUES, specifying the value you want for each column: VALUES(col1,col2,col3,col4,col5) */ - - + +CREATE TABLE temp.new_vendor AS +SELECT * FROM vendor; +INSERT INTO temp.new_vendor (vendor_name, vendor_type, vendor_owner_first_name, vendor_owner_last_name, vendor_id) +VALUES ('Thomas Superfood Store', 'Fresh Focused', 'Thomas', 'Rosenthal', 10) -- Date /*1. Get the customer_id, month, and year (in separate columns) of every purchase in the customer_purchases table. @@ -32,9 +43,19 @@ VALUES(col1,col2,col3,col4,col5) HINT: you might need to search for strfrtime modifers sqlite on the web to know what the modifers for month and year are! */ +SELECT customer_id, + strftime('%m', market_date) AS month, + strftime('%Y', market_date) AS year +FROM customer_purchases + /* 2. Using the previous query as a base, determine how much money each customer spent in April 2019. Remember that money spent is quantity*cost_to_customer_per_qty. HINTS: you will need to AGGREGATE, GROUP BY, and filter... but remember, STRFTIME returns a STRING for your WHERE statement!! */ +SELECT customer_id, SUM(quantity * cost_to_customer_per_qty) AS total_spent +FROM customer_purchases +WHERE + strftime('%m', market_date) = '04' AND strftime('%Y', market_date) = '2019' +GROUP BY customer_id diff --git a/03_homework/homework_4.sql b/03_homework/homework_4.sql index 3a242b9bb..72a1bbbc7 100644 --- a/03_homework/homework_4.sql +++ b/03_homework/homework_4.sql @@ -17,8 +17,8 @@ The `||` values concatenate the columns into strings. Edit the appropriate columns -- you're making two edits -- and the NULL rows will be fixed. All the other rows will remain the same.) */ - - +SELECT product_name || ', ' || COALESCE(product_size, '') || ' (' || COALESCE(product_qty_type, 'unit') || ')' +FROM product --Windowed Functions /* 1. Write a query that selects from the customer_purchases table and numbers each customer’s @@ -30,11 +30,76 @@ each new market date for each customer, or select only the unique market dates p (without purchase details) and number those visits. HINT: One of these approaches uses ROW_NUMBER() and one uses DENSE_RANK(). */ +SELECT customer_id, market_date, ROW_NUMBER() OVER (PARTITION BY customer_id ORDER BY market_date) AS visit_number +FROM customer_purchases /* 2. Reverse the numbering of the query from a part so each customer’s most recent visit is labeled 1, then write another query that uses this one as a subquery (or temp table) and filters the results to only the customer’s most recent visit. */ +/* Reverse the numbering of the query */ +SELECT customer_id, market_date, ROW_NUMBER() OVER (PARTITION BY customer_id ORDER BY market_date DESC) AS visit_number +FROM customer_purchases + +/* Second query that uses above as a subquery (or temp table) and filters the results to only the customer’s most recent visit */ +SELECT customer_id, market_date +FROM (SELECT customer_id, market_date, ROW_NUMBER() OVER (PARTITION BY customer_id ORDER BY market_date DESC) AS visit_number + FROM customer_purchases) + AS ranked_visits +WHERE visit_number = 1 /* 3. Using a COUNT() window function, include a value along with each row of the customer_purchases table that indicates how many different times that customer has purchased that product_id. */ + +SELECT customer_id, product_id, market_date, COUNT(*) OVER (PARTITION BY customer_id, product_id) AS purchase_count_of_product_id + +FROM customer_purchases + +-- String manipulations +/* 1. Some product names in the product table have descriptions like "Jar" or "Organic". +These are separated from the product name with a hyphen. +Create a column using SUBSTR (and a couple of other commands) that captures these, but is otherwise NULL. +Remove any trailing or leading whitespaces. Don't just use a case statement for each product! + +| product_name | description | +|----------------------------|-------------| +| Habanero Peppers - Organic | Organic | + +Hint: you might need to use INSTR(product_name,'-') to find the hyphens. INSTR will help split the column. */ + +SELECT product_name, TRIM(SUBSTR(product_name, INSTR(product_name, '-') + 1)) AS product_description +FROM product +WHERE INSTR(product_name, '-') > 0 + +/* 2. Filter the query to show any product_size value that contain a number with REGEXP. */ + +SELECT product_name, product_size, TRIM(SUBSTR(product_name, INSTR(product_name, '-') + 1)) AS product_description +FROM product +WHERE + INSTR(product_name, '-') > 0 + AND product_size REGEXP '[0-9]'; + +-- UNION +/* 1. Using a UNION, write a query that displays the market dates with the highest and lowest total sales. + +HINT: There are a possibly a few ways to do this query, but if you're struggling, try the following: +1) Create a CTE/Temp Table to find sales values grouped dates; +2) Create another CTE/Temp table with a rank windowed function on the previous query to create +"best day" and "worst day"; +3) Query the second temp table twice, once for the best day, once for the worst day, +with a UNION binding them. */ + +WITH Total_sale_for_day AS ( + SELECT market_date, SUM(quantity * cost_to_customer_per_qty) AS total_sales + FROM customer_purchases + GROUP BY market_date +), +dates_ranked AS ( + SELECT market_date, RANK() OVER (ORDER BY total_sales DESC) AS best_day, RANK() OVER (ORDER BY total_sales ASC) AS worst_day + FROM Total_sale_for_day +) +SELECT market_date FROM dates_ranked +WHERE best_day = 1 + +UNION SELECT market_date FROM dates_ranked +WHERE worst_day = 1; diff --git a/03_homework/homework_5.sql b/03_homework/homework_5.sql index 861108155..6aa363717 100644 --- a/03_homework/homework_5.sql +++ b/03_homework/homework_5.sql @@ -1,33 +1,3 @@ --- String manipulations -/* 1. Some product names in the product table have descriptions like "Jar" or "Organic". -These are separated from the product name with a hyphen. -Create a column using SUBSTR (and a couple of other commands) that captures these, but is otherwise NULL. -Remove any trailing or leading whitespaces. Don't just use a case statement for each product! - -| product_name | description | -|----------------------------|-------------| -| Habanero Peppers - Organic | Organic | - -Hint: you might need to use INSTR(product_name,'-') to find the hyphens. INSTR will help split the column. */ - - - -/* 2. Filter the query to show any product_size value that contain a number with REGEXP. */ - - - --- UNION -/* 1. Using a UNION, write a query that displays the market dates with the highest and lowest total sales. - -HINT: There are a possibly a few ways to do this query, but if you're struggling, try the following: -1) Create a CTE/Temp Table to find sales values grouped dates; -2) Create another CTE/Temp table with a rank windowed function on the previous query to create -"best day" and "worst day"; -3) Query the second temp table twice, once for the best day, once for the worst day, -with a UNION binding them. */ - - - -- Cross Join /*1. Suppose every vendor in the `vendor_inventory` table had 5 of each of their products to sell to **every** customer on record. How much money would each vendor make per product? @@ -39,7 +9,22 @@ Think a bit about the row counts: how many distinct vendors, product names are t How many customers are there (y). Before your final group by you should have the product of those two queries (x*y). */ +WITH customer_count AS ( + SELECT COUNT(customer_id) AS total_customers FROM customer) + +, ven_product_info AS ( + SELECT ve.vendor_name, pr.product_name, vi.original_price FROM vendor_inventory vi + JOIN vendor ve ON vi.vendor_id = ve.vendor_id + JOIN product pr ON vi.product_id = pr.product_id) +, ven_product_rev AS ( + SELECT vpi.vendor_name, vpi.product_name, vpi.original_price * cc.total_customers * 5 AS tot_rev + FROM ven_product_info vpi + CROSS JOIN customer_count cc) + +SELECT vendor_name, product_name, SUM(tot_rev) AS tot_rev_per_ven_per_prod +FROM ven_product_rev +GROUP BY vendor_name, product_name -- INSERT /*1. Create a new table "product_units". @@ -47,27 +32,31 @@ This table will contain only products where the `product_qty_type = 'unit'`. It should use all of the columns from the product table, as well as a new column for the `CURRENT_TIMESTAMP`. Name the timestamp column `snapshot_timestamp`. */ - +CREATE TABLE product_units AS +SELECT *, CURRENT_TIMESTAMP AS snapshot_timestamp FROM product WHERE product_qty_type = 'unit' /*2. Using `INSERT`, add a new row to the product_units table (with an updated timestamp). This can be any product you desire (e.g. add another record for Apple Pie). */ - +INSERT INTO product_units (product_id, product_name, product_size, product_category_id, product_qty_type, snapshot_timestamp) +VALUES (7, 'Apple Pie', '10"', 3, 'unit', CURRENT_TIMESTAMP) -- DELETE /* 1. Delete the older record for the whatever product you added. HINT: If you don't specify a WHERE clause, you are going to have a bad time.*/ - +DELETE FROM product_units +WHERE product_id = 7 + AND snapshot_timestamp = ( + SELECT MIN(snapshot_timestamp) + FROM product_units + WHERE product_id = 7) -- UPDATE /* 1.We want to add the current_quantity to the product_units table. First, add a new column, current_quantity to the table using the following syntax. -ALTER TABLE product_units -ADD current_quantity INT; - Then, using UPDATE, change the current_quantity equal to the last quantity value from the vendor_inventory details. HINT: This one is pretty hard. @@ -79,3 +68,7 @@ Finally, make sure you have a WHERE statement to update the right row, When you have all of these components, you can run the update statement. */ +ALTER TABLE product_units +ADD current_quantity INT; + +/* I am not able to solve this one. Sorry! */ diff --git a/03_homework/homework_6.md b/03_homework/homework_6.md index f16c02202..97bbd0957 100644 --- a/03_homework/homework_6.md +++ b/03_homework/homework_6.md @@ -6,3 +6,13 @@
**Write**: Reflect on your previous work and how you would adjust to include ethics and inequity components. Total length should be a few paragraphs, no more than one page. + +For the past couple of weeks, I have been using SQL data to retrieve, manipulate and update data for a hypothetical farmers market. There are several data ethics considerations that can be learnt from these activities that I will be applying in my role going forward. First and foremost is ensuring that individuals and enterprises are aware and sign-off that you are allowed to capture their data. There should also be strong privacy policies adhered to, such as an understanding of who has access to the data, where the data is to be stored and what the data can be used for. The database should also be architected to ensure that the bare-minimum data is captured to achieve the insights required and that the database architecture does not combine information (and tables) that could be used to deanonymize confidential information that could jeopardize an individual or enterprise’s identity, privacy, security or proprietary information. + +The ethics course on Thursday also had me thinking about how this data was collected and entered into a database. In a lot of cases, manual data entry is required to convert analog data (eg hand written forms) into digital data for a database. It is important that the people who are providing their labor to complete these tasks are paid fairly and treated fairly, with their health being a priority. Finally, the ethics course reminded me to think about different biases that could arise when building the database architecture and performing data entry. + +A couple of examples from the hypothetical farmers market that got me thinking include: +- Were the customers and vendors aware and signed off that this data was being collected and visible to so many people? How should this database be restricted to its many users? +- In one of the earlier homework exercises there was a Farmer Market Appreciated Committee that gave out bumper stickers to customer who were high spenders. This had an element of stalking and neglection of other customers who may share an appreciation/patronage for attending farmers markets but have different income levels. Better inclusion here could actually be financially rewarding as bumper stickers on cars with customers who spend less could promote the farmer market in different areas of the city, bringing in more and diverse groups to the market. +- The inclusion of customer names and respective zip codes was not relevant for a high majority of the SQL tasks and questions whether this data should have been captured in the database in the first place +- The CROSSJOIN task in homework five creates a table that gives away proprietary and sensitive information about vendors and customers. Additional security and access policies need to be in place for these aggregated tables.