diff --git a/02_activities/assignments/DC_Cohort/Assignment1-CL.drawio.png b/02_activities/assignments/DC_Cohort/Assignment1-CL.drawio.png new file mode 100644 index 000000000..d00a6ff4a Binary files /dev/null and b/02_activities/assignments/DC_Cohort/Assignment1-CL.drawio.png differ diff --git a/02_activities/assignments/DC_Cohort/Assignment1.md b/02_activities/assignments/DC_Cohort/Assignment1.md index f78778f5b..29ca068fc 100644 --- a/02_activities/assignments/DC_Cohort/Assignment1.md +++ b/02_activities/assignments/DC_Cohort/Assignment1.md @@ -205,5 +205,12 @@ Consider, for example, concepts of fariness, inequality, social structures, marg ``` -Your thoughts... +Values embedded in databases and data systems may directly influence the societal structures and individual experiences encountered in day-to-day life, often reflecting societal norms and biases. The Wired article, When Databases Get to Define Family, show examples of this which can happen in Pakistan but may also happen in nations with centralized and powerful governments like China. As such, the design choices and underlying logic of these systems can shape access, inclusion, and marginalization for these countries. + +Fortunately, to my knowledge and experiences as a Canadian Citizen, Canada and Ontario has not had such large blockages on services if data has not been updated or expired. Necessities like health care and everyday transit would still be available to you. However, some services or allowances like not having an updated driver’s license and driving is illegal or similarly having an outdated passport and flying internationally. In other cases, like the act of voting, showing proof of address from a large variety of sources ranging from the standard driver’s license to a school transcript is sufficient. This recognizes the right to vote for people that are in the area and enables those who may not have stable housing or access to official documents. + +An interesting case in my day-to-day life would be the marketing information and the profile that has been built up over time by Google that is sold to different advertisers. The amount of information that has been built up over time as a trade off for using their services is vast and scarily accurate as they have been tracking searches, emails, clicks, and so on over large periods of time. The amount of detail was unknown to me until I have specifically requested my information, and this is crazy because I believe that if more people knew what was tracked, they would be hesitant on what is agreed upon when clicking the “I agree” button. As such, data privacy, security, and integrity come into question as a simple leak could expose many details from a huge population and it gets easier with more companies paying for access to this information. + +Although I do not currently know of many examples of fairness, inequality, social structures, and marginalization in my day-to-day life, reading about these systems will change and affect how I think about and design data systems in the future. Strategies like allowing for future modifications in case laws or views change or having policy makers or committees overseeing some aspects of data systems may be a good start. + ``` diff --git a/02_activities/assignments/DC_Cohort/Assignment2.md b/02_activities/assignments/DC_Cohort/Assignment2.md index 9b804e9ee..894f35bde 100644 --- a/02_activities/assignments/DC_Cohort/Assignment2.md +++ b/02_activities/assignments/DC_Cohort/Assignment2.md @@ -183,5 +183,10 @@ Consider, for example, concepts of labour, bias, LLM proliferation, moderating c ``` -Your thoughts... +The story “Neural nets are just people all the way down” highlights several crucial ethical issues at the intersection of machine learning, labor, bias, content moderation, and broader technology-society impacts. The article makes it clear that, beneath the surface of modern AI and large language models, extensive manual human labor and subjective decision-making profoundly shape these systems and their outcomes. + +Many to all of the AI/neural net services are built on the backs on underpaid and overworked people who do not receive any recognition. This would be similar to how things are manufactured like clothes and other consumer goods. People from more marginalized backgrounds or have some power imbalance are a huge part of both these processes. Additionally, since the backbone databases of these services (ImageNet and others) have leveraged human identification or classification, a person’s opinions, biases, and judgements would come into play when labeling certain images. Examples being words to describe a person or associations to items in order to build their word nets. These biases also would not include people’s own incentives. If there were bad faith actors or people who were trying to sort through the most to get paid, things may not be labelled as accurately as people who would take their time and think about each item. + +It is very important to think about the use and generation of AI-related content as it has been built on all these prior classifications – the well-intentioned and those of ill will. This means that the things generated from these AI services should be looked at with a critical eye before using or releasing the products to the public. A recent example of little to no review of AI content would be Coca Cola. They have been under fire as they decided to use AI to generate a commercial riddled with errors instead of paying artists or film crews to make a good piece of media. As a result, they have been heavily ridiculed online and fortunately brought some light into the current capabilities of AI - useful to generate base ideas and flesh out processes but ultimately still requires a human touch to finish a product. + ``` diff --git a/02_activities/assignments/DC_Cohort/assignment-twoSQL.drawio.png b/02_activities/assignments/DC_Cohort/assignment-twoSQL.drawio.png new file mode 100644 index 000000000..0da63a2f9 Binary files /dev/null and b/02_activities/assignments/DC_Cohort/assignment-twoSQL.drawio.png differ diff --git a/02_activities/assignments/DC_Cohort/assignment1.sql b/02_activities/assignments/DC_Cohort/assignment1.sql index c992e3205..3e4c4a93e 100644 --- a/02_activities/assignments/DC_Cohort/assignment1.sql +++ b/02_activities/assignments/DC_Cohort/assignment1.sql @@ -4,17 +4,19 @@ --SELECT /* 1. Write a query that returns everything in the customer table. */ - +SELECT * FROM customer; /* 2. Write a query that displays all of the columns and 10 rows from the cus- tomer table, sorted by customer_last_name, then customer_first_ name. */ - +SELECT * FROM customer +ORDER BY customer_last_name, customer_first_name +LIMIT 10; --WHERE /* 1. Write a query that returns all customer purchases of product IDs 4 and 9. */ - +SELECT * FROM customer_purchases WHERE product_id IN (4, 9); /*2. Write a query that returns all customer purchases and a new calculated column 'price' (quantity * cost_to_customer_per_qty), @@ -23,10 +25,18 @@ filtered by customer IDs between 8 and 10 (inclusive) using either: 2. one condition using BETWEEN */ -- option 1 - +SELECT + *, + (quantity * cost_to_customer_per_qty) AS price +FROM customer_purchases +WHERE customer_id >= 8 AND customer_id <= 10; -- option 2 - +SELECT + *, + (quantity * cost_to_customer_per_qty) AS price +FROM customer_purchases +WHERE customer_id BETWEEN 8 AND 10; --CASE @@ -35,20 +45,41 @@ Using the product table, write a query that outputs the product_id and product_n columns and add a column called prod_qty_type_condensed that displays the word “unit” if the product_qty_type is “unit,” and otherwise displays the word “bulk.” */ - +SELECT + product_id, + product_name, + CASE + WHEN product_qty_type = 'unit' THEN 'unit' + ELSE 'bulk' + END AS prod_qty_type_condensed +FROM product; /* 2. We want to flag all of the different types of pepper products that are sold at the market. add a column to the previous query called pepper_flag that outputs a 1 if the product_name contains the word “pepper” (regardless of capitalization), and otherwise outputs 0. */ - +SELECT + product_id, + product_name, + CASE + WHEN product_name LIKE '%pepper%' THEN 1 + ELSE 0 + END as pepper_flag +FROM product; --JOIN /* 1. Write a query that INNER JOINs the vendor table to the vendor_booth_assignments table on the vendor_id field they both have in common, and sorts the result by vendor_name, then market_date. */ - - +SELECT + * +FROM + vendor + INNER JOIN vendor_booth_assignments + ON vendor.vendor_id = vendor_booth_assignments.vendor_id +ORDER BY + vendor_name, + market_date; /* SECTION 3 */ @@ -56,7 +87,11 @@ vendor_id field they both have in common, and sorts the result by vendor_name, t /* 1. Write a query that determines how many times each vendor has rented a booth at the farmer’s market by counting the vendor booth assignments per vendor_id. */ - +SELECT + vendor_id, + COUNT(vendor_id) +FROM vendor_booth_assignments +GROUP BY vendor_id; /* 2. The Farmer’s Market Customer Appreciation Committee wants to give a bumper sticker to everyone who has ever spent more than $2000 at the market. Write a query that generates a list @@ -64,6 +99,20 @@ of customers for them to give stickers to, sorted by last name, then first name. HINT: This query requires you to join two tables, use an aggregate function, and use the HAVING keyword. */ +SELECT + customer.customer_id, + customer.customer_last_name, + customer.customer_first_name, + SUM(quantity * cost_to_customer_per_qty) AS total_spent +FROM customer_purchases + INNER JOIN customer + ON customer_purchases.customer_id = customer.customer_id +GROUP BY + customer.customer_id +HAVING total_spent > 2000 +ORDER BY + customer_last_name, + customer_first_name; --Temp Table @@ -79,6 +128,13 @@ VALUES(col1,col2,col3,col4,col5) */ +CREATE TEMP TABLE new_vendor AS +SELECT * FROM vendor; +INSERT INTO new_vendor VALUES(10, 'Thomass Superfood Store', 'Fresh Focused', 'Thomas', 'Rosenthal'); +-- View temp table to verify +SELECT * FROM new_vendor; + + -- Date /*1. Get the customer_id, month, and year (in separate columns) of every purchase in the customer_purchases table. @@ -86,6 +142,11 @@ VALUES(col1,col2,col3,col4,col5) HINT: you might need to search for strfrtime modifers sqlite on the web to know what the modifers for month and year are! */ +SELECT + customer_id, + STRFTIME('%m', market_date) AS purchase_month, + STRFTIME('%Y', market_date) AS purchase_year +FROM customer_purchases; /* 2. Using the previous query as a base, determine how much money each customer spent in April 2022. @@ -94,3 +155,10 @@ Remember that money spent is quantity*cost_to_customer_per_qty. HINTS: you will need to AGGREGATE, GROUP BY, and filter... but remember, STRFTIME returns a STRING for your WHERE statement!! */ +SELECT + customer_id, + SUM(quantity*cost_to_customer_per_qty) AS total_spent +FROM customer_purchases +WHERE STRFTIME('%m', market_date) = '04' AND STRFTIME('%Y', market_date) = '2022' +GROUP BY customer_id; + diff --git a/02_activities/assignments/DC_Cohort/assignment2.sql b/02_activities/assignments/DC_Cohort/assignment2.sql index 5ad40748a..6116b6dc1 100644 --- a/02_activities/assignments/DC_Cohort/assignment2.sql +++ b/02_activities/assignments/DC_Cohort/assignment2.sql @@ -20,7 +20,18 @@ The `||` values concatenate the columns into strings. Edit the appropriate columns -- you're making two edits -- and the NULL rows will be fixed. All the other rows will remain the same.) */ +-- Find nulls +SELECT + product_name, + product_size, + product_qty_type +FROM product +WHERE product_size IS NULL OR product_qty_type IS NULL; +-- Coalesce with null replacements +SELECT + product_name || ', ' || COALESCE(product_size, '') || ' (' || COALESCE(product_qty_type, 'unit') || ')' +FROM product; --Windowed Functions /* 1. Write a query that selects from the customer_purchases table and numbers each customer’s @@ -32,18 +43,68 @@ each new market date for each customer, or select only the unique market dates p (without purchase details) and number those visits. HINT: One of these approaches uses ROW_NUMBER() and one uses DENSE_RANK(). */ - +SELECT + customer_id, + market_date, + ROW_NUMBER() OVER (PARTITION BY customer_id ORDER BY market_date) AS visit_number +FROM ( + SELECT DISTINCT customer_id, market_date + FROM customer_purchases +) AS unique_visits +ORDER BY customer_id, market_date; + +SELECT + customer_id, + market_date, + DENSE_RANK() OVER (PARTITION BY customer_id ORDER BY market_date) AS visit_number +FROM ( + SELECT DISTINCT customer_id, market_date + FROM customer_purchases +) AS unique_visits +ORDER BY customer_id, market_date; /* 2. Reverse the numbering of the query from a part so each customer’s most recent visit is labeled 1, then write another query that uses this one as a subquery (or temp table) and filters the results to only the customer’s most recent visit. */ - +-- Reverse numbering of visits to find most recent visit +SELECT + customer_id, + market_date, + ROW_NUMBER() OVER (PARTITION BY customer_id ORDER BY market_date DESC) AS visit_number +FROM ( + SELECT DISTINCT customer_id, market_date + FROM customer_purchases +) AS unique_visits +ORDER BY customer_id, market_date DESC; + +-- Subquery to filter most recent visit +SELECT + customer_id, + market_date, + visit_number +FROM ( + SELECT + customer_id, + market_date, + ROW_NUMBER() OVER (PARTITION BY customer_id ORDER BY market_date DESC) AS visit_number + FROM ( + SELECT DISTINCT customer_id, market_date + FROM customer_purchases + ) AS unique_visits +) AS ranked_visits +WHERE visit_number = 1 +ORDER BY customer_id; /* 3. Using a COUNT() window function, include a value along with each row of the customer_purchases table that indicates how many different times that customer has purchased that product_id. */ - +SELECT + customer_id, + product_id, + market_date, + COUNT() OVER (PARTITION BY customer_id, product_id) AS purchase_count +FROM customer_purchases; -- String manipulations /* 1. Some product names in the product table have descriptions like "Jar" or "Organic". @@ -57,11 +118,21 @@ Remove any trailing or leading whitespaces. Don't just use a case statement for Hint: you might need to use INSTR(product_name,'-') to find the hyphens. INSTR will help split the column. */ - +SELECT + product_name, + CASE + WHEN INSTR(product_name, '-') > 0 THEN TRIM(SUBSTR(product_name, INSTR(product_name, '-') + 1)) + ELSE NULL + END AS description +FROM product; /* 2. Filter the query to show any product_size value that contain a number with REGEXP. */ - +SELECT + product_name, + product_size +FROM product +WHERE product_size REGEXP '\d'; -- UNION /* 1. Using a UNION, write a query that displays the market dates with the highest and lowest total sales. @@ -73,8 +144,33 @@ HINT: There are a possibly a few ways to do this query, but if you're struggling 3) Query the second temp table twice, once for the best day, once for the worst day, with a UNION binding them. */ - - +WITH sales_per_date AS ( + SELECT + market_date, + SUM(quantity*cost_to_customer_per_qty) AS total_sales + FROM customer_purchases + GROUP BY market_date +), ranked_sales AS ( + SELECT + market_date, + total_sales, + -- Find max and min sales via the RANK window + RANK() OVER (ORDER BY total_sales DESC) AS sales_rank_desc, + RANK() OVER (ORDER BY total_sales ASC) AS sales_rank_asc + FROM sales_per_date +) +-- Join the max and min found +SELECT + market_date, + total_sales +FROM ranked_sales +WHERE sales_rank_desc = 1 +UNION +SELECT + market_date, + total_sales +FROM ranked_sales +WHERE sales_rank_asc = 1; /* SECTION 3 */ @@ -90,6 +186,34 @@ How many customers are there (y). Before your final group by you should have the product of those two queries (x*y). */ +--Come back to this to rethink and test. Not clear on cross join usage here +SELECT + v.vendor_name, + p.product_name, + vi.original_price * 5 * COUNT(DISTINCT c.customer_id) AS total_potential_revenue +FROM vendor_inventory AS vi +CROSS JOIN customer AS c + JOIN vendor AS v ON vi.vendor_id = v.vendor_id + JOIN product AS p ON vi.product_id = p.product_id +GROUP BY v.vendor_name, p.product_name, vi.original_price; + +/* +-- Testing cross join +SELECT +* +FROM vendor_inventory AS vi +CROSS JOIN customer; + +-- Alternative without cross join +SELECT + v.vendor_name, + p.product_name, + vi.original_price * 5 * (SELECT COUNT(*) FROM customer) AS total_potential_revenue +FROM vendor_inventory vi +JOIN vendor v ON vi.vendor_id = v.vendor_id +JOIN product p ON vi.product_id = p.product_id +GROUP BY v.vendor_name, p.product_name, vi.original_price; +*/ -- INSERT /*1. Create a new table "product_units". @@ -97,18 +221,31 @@ This table will contain only products where the `product_qty_type = 'unit'`. It should use all of the columns from the product table, as well as a new column for the `CURRENT_TIMESTAMP`. Name the timestamp column `snapshot_timestamp`. */ - +CREATE TABLE product_units AS +SELECT + product_id, + product_name, + product_size, + product_qty_type, + CURRENT_TIMESTAMP AS snapshot_timestamp +FROM product +WHERE product_qty_type = 'unit'; /*2. Using `INSERT`, add a new row to the product_units table (with an updated timestamp). This can be any product you desire (e.g. add another record for Apple Pie). */ - +INSERT INTO product_units + (product_id, product_name, product_size, product_qty_type, snapshot_timestamp) +VALUES + (1234, 'Test123', '1 unit', 'unit', CURRENT_TIMESTAMP); -- DELETE /* 1. Delete the older record for the whatever product you added. HINT: If you don't specify a WHERE clause, you are going to have a bad time.*/ +DELETE FROM product_units +WHERE product_id = 1234; -- UPDATE @@ -128,6 +265,32 @@ Finally, make sure you have a WHERE statement to update the right row, you'll need to use product_units.product_id to refer to the correct row within the product_units table. When you have all of these components, you can run the update statement. */ +ALTER TABLE product_units +ADD current_quantity INT; - +UPDATE product_units +SET current_quantity = COALESCE(( + -- Subquery to find the quantity for each product + SELECT vi.quantity + FROM vendor_inventory AS vi + WHERE vi.product_id = product_units.product_id + -- Finding the latest date for the product + ORDER BY vi.market_date DESC + LIMIT 1 +), 0); + +/* +UPDATE product_units +SET current_quantity = COALESCE(( + SELECT + vi.quantity + FROM vendor_inventory AS vi + WHERE vi.product_id = product_units.product_id + -- Find the latest date for the product by matching the product id with the product_id from the current date + AND vi.market_date = ( + SELECT MAX(vi2.market_date) + FROM vendor_inventory AS vi2 + WHERE vi2.product_id = vi.product_id + ) +), 0);