05 Jun

MySQL: Cheesy Way To Row Number

The classic way to create row numbers in MySQL is to use a variable and increment it for each row.  I did an example earlier showing it this way.  In SQL Server, you just use ROW_NUMBER() or one of its variants.  But, just for fun here is another way which was in the SQL Cookbook.

For this I’ll be using my good old military spending data.

mysql> select @ctr := @ctr + 1 AS rowCtr, region, country, spending
    -> FROM militaryspending
    -> JOIN (SELECT @ctr := 0) AS a
    -> ORDER BY spending DESC LIMIT 5;
+--------+---------------+----------------+----------+
| rowCtr | region        | country        | spending |
+--------+---------------+----------------+----------+
|      1 | North America | United States  |   711421 |
|      2 | Asia          | China          |   142859 |
|      3 | Asia          | Russia         |    71853 |
|      4 | Europe        | United Kingdom |    62685 |
|      5 | Europe        | France         |    62535 |
+--------+---------------+----------------+----------+
5 rows in set (0.00 sec)

Anyways, that is pretty easy. Suppose, however, that we want to see the last 5. We can do this, but it won’t number the way we, or I, want it to.

mysql> select @ctr := @ctr + 1 AS rowCtr, region, country, spending
    -> FROM militaryspending
    -> JOIN (SELECT @ctr := 0) AS a
    -> ORDER BY spending LIMIT 5;
+--------+---------------+------------+----------+
| rowCtr | region        | country    | spending |
+--------+---------------+------------+----------+
|      1 | Africa        | Seychelles |      9.3 |
|      2 | Africa        | Cape Verde |      9.7 |
|      3 | Africa        | Mauritius  |     10.1 |
|      4 | North America | Belize     |     15.7 |
|      5 | Europe        | Moldova    |     20.8 |
+--------+---------------+------------+----------+
5 rows in set (0.00 sec)

I, actually, wanted to see it numbered as 126, 125, 124, etc.

That’s actually easy to do, just do a subquery / inline view as follows:

mysql> SELECT rowCtr, region, country, spending
    -> FROM
    -> (
    -> select @ctr := @ctr + 1 AS rowCtr, region, country, spending
    -> FROM militaryspending
    -> JOIN (SELECT @ctr := 0) AS a
    -> ORDER BY spending DESC
    -> ) AS b
    -> ORDER BY rowCtr DESC LIMIT 5;
+--------+---------------+------------+----------+
| rowCtr | region        | country    | spending |
+--------+---------------+------------+----------+
|    126 | Africa        | Seychelles |      9.3 |
|    125 | Africa        | Cape Verde |      9.7 |
|    124 | Africa        | Mauritius  |     10.1 |
|    123 | North America | Belize     |     15.7 |
|    122 | Europe        | Moldova    |     20.8 |
+--------+---------------+------------+----------+
5 rows in set (0.00 sec)

The SQL Cookbook also had an interesting approach, they used a COUNT(*). It’s a weird solution because it can only work on an ordered list of some sort. For example, if we alphabetized our list of countries it would work as follows:

mysql> SELECT a.country,
    -> a.spending,
    -> (SELECT COUNT(*) FROM militaryspending WHERE country <= a.country) as rowCtr     
    -> FROM militaryspending a
    -> ORDER BY country LIMIT 5;
+-------------+----------+--------+
| country     | spending | rowCtr |
+-------------+----------+--------+
| Afghanistan |      878 |      1 |
| Albania     |      197 |      2 |
| Algeria     |     8665 |      3 |
| Angola      |     3647 |      4 |
| Argentina   |     3295 |      5 |
+-------------+----------+--------+
5 rows in set (0.01 sec)

I dunno, weird solution but it’s what he did, and it does work in some situations. I read that section really quick so maybe I missed something, something really obvious but I’ll stick to the variable for now.

28 May

SAS: Constraints

This will be ugly.  I’m looking at the Study Guide and it does a terrible job on constraints.  For instance, it mentions referential integrity (foreign keys) but it does not show an example.  However, it also lists several keywords that would apply to it.  So, do I need to know it for the test?

There are about 6 pages on the guide so I’m going to try and dig out what I think I would need to know.  Since I already know most of this, from other databases, I’ll surely skip useful parts.  And, no way am I doing each database version. This post is only about SAS.

Anyways, if it helps, great,  If it doesn’t, well, you got what you paid for, I guess.

CHECK

A check constraint is similar to a WHERE clause but it’s applied at the column level.  For example,

proc sql;
CREATE TABLE work.myjunk
(
	name char(10) CHECK (name IN ('Billy','Bob','Thornton')),
	number int CHECK (number < 10)
);
DESCRIBE TABLE CONSTRAINTS work.myjunk;
quit;     Integrity              Where
#    Constraint    Type     Clause
----------------------------------------------------------------
1    _CK0001_      Check    name in ('Billy', 'Bob', 'Thornton')
2    _CK0002_      Check    number

This syntax creates two CHECK constraints:

  • name: It must be one of Billy, Bob or Thornton.
  • number: It must be less than 10.

Because we didn’t name the constraints, SAS assigned a default name to them. I’m not going to list them as they’ll be in the sample code for each constraint. You can also name a constraint through the ALTER TABLE statement or adding the constraint to the end of the CREATE TABLE syntax as follows:

proc sql;
CREATE TABLE work.myjunk
(
	name char(10),
	number int,
	CONSTRAINT mycheck CHECK (number < 10)
);

DESCRIBE TABLE CONSTRAINTS work.myjunk;
quit;
     Integrity               Where
#    Constraint     Type     Clause
-----------------------------------------------------------------
1    mycheck        Check    number

I won’t do this every time. This is already going to be a very long, and it’s already disorganized enough, but I’ll try to sprinkle the variations throughout the post so you are comfortable seeing them both ways.

A CHECK constraint can also apply to a different column in the same row.

proc sql;
CREATE TABLE work.myjunk
(
	number int,
	number1 int CHECK (number1 < number)
);
DESCRIBE TABLE CONSTRAINTS work.myjunk;
quit;

     Integrity              Where
#    Constraint    Type     Clause
------------------------------------------
1    _CK0001_      Check    number1<number

This is precisely the type of thing I would expect SAS to ask you on a test.

NOT NULL

NOT NULL simply says no NULL, or missing (this is SAS) values, in the column. You write the constraint as follows:

proc sql;
CREATE TABLE work.myjunk
(
	name char(10) NOT NULL,
	number int,
	CONSTRAINT myconstraint NOT NULL (number)
);
DESCRIBE TABLE CONSTRAINTS work.myjunk;
quit;

     Integrity
#    Constraint      Type        Variables
------------------------------------------
1    _NM0001_        Not Null    name     
2    myconstraint    Not Null    number

I thought you could add this with an ALTER TABLE / MODIFY statement but apparently not.

proc sql;
CREATE TABLE work.myjunk
(
	name char(10) NOT NULL,
	number int
);
ALTER TABLE work.myjunk MODIFY number int NOT NULL;
DESCRIBE TABLE CONSTRAINTS work.myjunk;
quit;

     Integrity
#    Constraint    Type        Variables
----------------------------------------
1    _NM0001_      Not Null    name

To be honest, I was sure it, would work. In MySQL:

mysql> CREATE TABLE work.myjunk
    -> (
    ->  name char(10) NOT NULL,
    ->  number int
    -> );
Query OK, 0 rows affected (0.11 sec)

mysql> ALTER TABLE work.myjunk MODIFY number int NOT NULL;
Query OK, 0 rows affected (0.28 sec)
Records: 0  Duplicates: 0  Warnings: 0

mysql> desc work.myjunk;
+--------+----------+------+-----+---------+-------+
| Field  | Type     | Null | Key | Default | Extra |
+--------+----------+------+-----+---------+-------+
| name   | char(10) | NO   |     | NULL    |       |
| number | int(11)  | NO   |     | NULL    |       |
+--------+----------+------+-----+---------+-------+

The Null column shows NO for Nulls so it worked just fine on that side of the house.  I guess the answer is to only define a NOT NULL constraint in the CREATE TABLE statement for SAS.  You should probably also make a mental note of _NM0001_.  The automatic naming policy of SAS uses NM, not NN, for NOT NULL.  Yeah, that’s logical.  I suppose that’s how they keep the trick questions flowing.

UNIQUE

This constraint simply specifies that all values in the column must be unique. The study guide doesn’t mention this but UNIQUE usually allows for a NULL / missing value, but, only one as two would violate the constraint.

proc sql;
CREATE TABLE work.myjunk
(
	name char(10) UNIQUE,
	number int
);
DESCRIBE TABLE CONSTRAINTS work.myjunk;
quit;

     Integrity
#    Constraint    Type      Variables
--------------------------------------
1    _UN0001_      Unique    name

The constraint seems to work similarly to the NOT NULL constraint in terms of how you define it, however, you may want to apply it to multiple columns as follows:

proc sql;
CREATE TABLE work.myjunk
(
	name char(10),
	number int,
	CONSTRAINT myconstraint unique (number, name)
);
DESCRIBE TABLE CONSTRAINTS work.myjunk;
quit;
     Integrity
#    Constraint      Type      Variables
------------------------------------------
1    myconstraint    Unique    number name

PRIMARY KEY

SAS defines the PRIMARY KEY as a combination of the UNIQUE and NOT NULL constraints. I’m assuming, and have not verified this, yet, that unlike SQL Server, or INNODB tables in MySQL that it is not clustered. However, I haven’t hit the index part of the Study Guide so SAS may do something like that.

There can only be one primary key on a table. ALL of the following can be used to create a primary key.

proc sql;
CREATE TABLE work.myjunk
(
	name char(10),
	number int PRIMARY KEY
);
DESCRIBE TABLE CONSTRAINTS work.myjunk;
quit;

     Integrity
#    Constraint    Type           Variables
-------------------------------------------
1    _PK0001_      Primary Key    number

proc sql;
CREATE TABLE work.myjunk
(
name char(10),
number int,
CONSTRAINT mypk PRIMARY KEY (number)
);
DESCRIBE TABLE CONSTRAINTS work.myjunk;
quit;

Integrity
# Constraint Type Variables
-------------------------------------------
1 mypk Primary Key number

proc sql;
CREATE TABLE work.myjunk
(
name char(10),
number int,
CONSTRAINT mypk PRIMARY KEY (name,number)
);
DESCRIBE TABLE CONSTRAINTS work.myjunk;
quit;

     Integrity
#    Constraint    Type           Variables
---------------------------------------------
1    mypk          Primary Key    name number

The last version specifies that both the name and number fields are part of the primary key and that the restrictions for a primary key apply to both columns. For instance, the combination of the two columns must be unique, but you can repeat a value in a column. For example, this is ok,

name  number
A     1
A     2

But this would not be,

name  number
A     1
A     1

This also applies to a unique constraint that isn’t a primary key.

I’m going to stop now. I find myself getting a bit mixed up on the syntax with all the copy/pasting I’m doing and I fear I’m going to blow it. I think, however, this should give a decent starting point to constraints even if it feels woefully incomplete to me.

27 May

Inserting with a SELECT

Honestly, I don’t think I’ve ever done this outside of a book, but, it’s something that comes up so it’s worth mentioning.  Fortunately, everyone happily works the same way so I’m just going to do this in MySQL.

Suppose we have two identical tables and we want to copy data from one to another. We could use an external tool, like BCP, or mysqldump, or we could just use a INSERT / SELECT as follows;

mysql> DESC myjunk;
+——–+———+——+—–+———+——-+
| Field | Type | Null | Key | Default | Extra |
+——–+———+——+—–+———+——-+
| name | char(1) | NO | | NULL | |
| number | int(11) | NO | | 0 | |
+——–+———+——+—–+———+——-+
2 rows in set (0.00 sec)

mysql> DESC myjunk1;
+——–+———+——+—–+———+——-+
| Field | Type | Null | Key | Default | Extra |
+——–+———+——+—–+———+——-+
| name | char(1) | NO | | NULL | |
| number | int(11) | NO | | 0 | |
+——–+———+——+—–+———+——-+
2 rows in set (0.00 sec)

mysql> SELECT * FROM myjunk;
+——+——–+
| name | number |
+——+——–+
| A | 11 |
| B | 22 |
| C | 33 |
+——+——–+
3 rows in set (0.00 sec)

mysql> SELECT * FROM myjunk1;
Empty set (0.00 sec)

mysql> INSERT INTO myjunk1
-> SELECT * FROM myjunk;
Query OK, 3 rows affected (0.02 sec)
Records: 3 Duplicates: 0 Warnings: 0

mysql> SELECT * FROM myjunk1;
+——+——–+
| name | number |
+——+——–+
| A | 11 |
| B | 22 |
| C | 33 |
+——+——–+
3 rows in set (0.00 sec)

This, thankfully, is super clean and for the most part it just works. However, there are a couple of tweaks to it.

mysql> INSERT INTO myjunk1
    -> SELECT number, name FROM myjunk;
ERROR 1406 (22001): Data too long for column 'name' at row 1

This was caused because the Name column is a char(1) so it can’t handle the longer number types.

mysql> ALTER TABLE myjunk1 MODIFY name varchar(10);
Query OK, 0 rows affected (0.17 sec)
Records: 0  Duplicates: 0  Warnings: 0

mysql> INSERT INTO myjunk1
    -> SELECT number, name FROM myjunk;
ERROR 1366 (HY000): Incorrect integer value: 'A' for column 'number' at row 1

It would have been nice if it had dropped this error first, before I modified the column, but MySQL isn’t know for solid/clean errors. Anyways, just make sure that if you are switching columns around, for some reason, that you don’t bonk the data types.

Also, you must have an equal number of columns. For example, this will not work.

mysql> desc myjunk1;
+--------+-------------+------+-----+---------+-------+
| Field  | Type        | Null | Key | Default | Extra |
+--------+-------------+------+-----+---------+-------+
| name   | varchar(10) | YES  |     | NULL    |       |
| number | int(11)     | NO   |     | 0       |       |
| newCol | varchar(10) | YES  |     | NULL    |       |
+--------+-------------+------+-----+---------+-------+
3 rows in set (0.00 sec)

mysql> desc myjunk;
+--------+---------+------+-----+---------+-------+
| Field  | Type    | Null | Key | Default | Extra |
+--------+---------+------+-----+---------+-------+
| name   | char(1) | NO   |     | NULL    |       |
| number | int(11) | NO   |     | 0       |       |
+--------+---------+------+-----+---------+-------+
2 rows in set (0.00 sec)

mysql> INSERT INTO myjunk1
    -> SELECT * FROM myjunk;
ERROR 1136 (21S01): Column count doesn't match value count at row 1

This, correctly, tossed an error in SQL Server, SAS and MySQL.

Good luck, and hope it helps someone, someday.

26 May

CREATE TABLE AS

Creating a table is, more or less, similar among the three variants that I write about.  You create a table and you roll with it.  However, the CTAS statement (CREATE TABLE AS) is handled somewhat differently amongst them.  So, I’m going to hit that one up.

Really, I’m just writing this up because outside of MySQL I always forget this syntax. Also, when you create a table this way, it won’t include

MySQL

Create a copy of a a table with all of its columns included.

mysql> CREATE TABLE myjunk1 AS SELECT * FROM myjunk;
Query OK, 3 rows affected (0.08 sec)
Records: 3  Duplicates: 0  Warnings: 0

mysql> SELECT * FROM myjunk1;
+------+--------+
| name | number |
+------+--------+
| A    |     11 |
| B    |     22 |
| C    |     33 |
+------+--------+
3 rows in set (0.00 sec)

Create a copy of a table, however, do not include any of its columns.

mysql> CREATE TABLE myjunk1 AS SELECT * FROM myjunk WHERE 1 = 2;
Query OK, 0 rows affected (0.12 sec)
Records: 0  Duplicates: 0  Warnings: 0

mysql> SELECT * FROM myjunk1;
Empty set (0.00 sec)

mysql> DESC myjunk1;
+--------+---------+------+-----+---------+-------+
| Field  | Type    | Null | Key | Default | Extra |
+--------+---------+------+-----+---------+-------+
| name   | char(1) | YES  |     | NULL    |       |
| number | int(11) | YES  |     | NULL    |       |
+--------+---------+------+-----+---------+-------+
2 rows in set (0.00 sec)

Also constraints are not included, for example.

mysql> ALTER TABLE myjunk ADD PRIMARY KEY(number);
Query OK, 3 rows affected (0.19 sec)
Records: 3  Duplicates: 0  Warnings: 0

mysql> CREATE TABLE myjunk1 AS SELECT * FROM myjunk;
Query OK, 3 rows affected (0.08 sec)
Records: 3  Duplicates: 0  Warnings: 0

mysql> DESC myjunk1;
+--------+---------+------+-----+---------+-------+
| Field  | Type    | Null | Key | Default | Extra |
+--------+---------+------+-----+---------+-------+
| name   | char(1) | YES  |     | NULL    |       |
| number | int(11) | NO   |     | 0       |       |
+--------+---------+------+-----+---------+-------+
2 rows in set (0.01 sec)

mysql> desc myjunk;
+--------+---------+------+-----+---------+-------+
| Field  | Type    | Null | Key | Default | Extra |
+--------+---------+------+-----+---------+-------+
| name   | char(1) | YES  |     | NULL    |       |
| number | int(11) | NO   | PRI | 0       |       |
+--------+---------+------+-----+---------+-------+
2 rows in set (0.00 sec)

Note that the primary key is not included in myjunk1.

I could go on but this is to help me remember the syntax.  You can read more here:

http://dev.mysql.com/doc/refman/5.6/en/create-table.html

SQL Server

Conceptually, SQL Server works in a similar fashion to MySQL, except, that they have their own syntax for the process. This won’t work:

CREATE TABLE myjunk1 AS SELECT * FROM myjunk;

Msg 156, Level 15, State 1, Line 1
Incorrect syntax near the keyword 'SELECT'.

Instead you need do something like this:

SELECT *
INTO myjunk1
FROM myjunk
WHERE number > 22;
SELECT * FROM myjunk1;

Name	Number
C	33

I had a really hard time remembering this syntax at first because I was so used to the CTAS variant.

http://msdn.microsoft.com/en-us/library/ms188029.aspx

SAS

SAS allows the CTAS variant, so I won’t repeat that but it also does something else. When you are creating a copy of a table, but don’t want to include any of it’s data, you can do the following:

proc sql;
CREATE TABLE work.myjunk
(
	name char(1),
	number int
);
INSERT INTO work.myjunk VALUES ('A',1)
                        VALUES ('B',2)
                        VALUES ('C',3);
CREATE TABLE work.myjunk1
LIKE work.myjunk;

DESCRIBE TABLE work.myjunk1;
quit;

From the log.

31         DESCRIBE TABLE work.myjunk1;
NOTE: SQL table WORK.MYJUNK1 was created like:

create table WORK.MYJUNK1( bufsize=131072 )
  (
   name char(1),
   number num
  );

Constraints, more or less, appear to follow the same rules as they do in the other variants.

All Variants – Adding A New Column

You can also create a new column simply by adding it to the result set. Be careful about using apostrophes to name the new column as it might not work.

mysql> CREATE TABLE myjunk1 AS
    -> SELECT name, number, 123 AS newCol
    -> FROM myjunk;
Query OK, 3 rows affected (0.10 sec)
Records: 3  Duplicates: 0  Warnings: 0

mysql> DESCRIBE myjunk1;
+--------+---------+------+-----+---------+-------+
| Field  | Type    | Null | Key | Default | Extra |
+--------+---------+------+-----+---------+-------+
| name   | char(1) | NO   |     | NULL    |       |
| number | int(11) | NO   |     | 0       |       |
| newCol | int(3)  | NO   |     | 0       |       |
+--------+---------+------+-----+---------+-------+
3 rows in set (0.00 sec)

mysql> SELECT * FROM myjunk1;
+------+--------+--------+
| name | number | newCol |
+------+--------+--------+
| A    |     11 |    123 |
| B    |     22 |    123 |
| C    |     33 |    123 |
+------+--------+--------+
3 rows in set (0.00 sec)

You can also CAST / CONVERT the data type. For example.

mysql> CREATE TABLE myjunk1 AS
    -> SELECT name, number, CAST(123 AS char) AS newCol
    -> FROM myjunk;
Query OK, 3 rows affected (0.12 sec)
Records: 3  Duplicates: 0  Warnings: 0

mysql> DESC myjunk1;
+--------+------------+------+-----+---------+-------+
| Field  | Type       | Null | Key | Default | Extra |
+--------+------------+------+-----+---------+-------+
| name   | char(1)    | NO   |     | NULL    |       |
| number | int(11)    | NO   |     | 0       |       |
| newCol | varchar(3) | YES  |     | NULL    |       |
+--------+------------+------+-----+---------+-------+
3 rows in set (0.00 sec)

I’m pretty sure that SAS doesn’t work that way but I’m not ready to write that today as I’m still learning that side of the house.  Another day, another post.

25 May

SAS: INSERT / VALUES

Originally, I’d planned this post to be about using CASE in an UPDATE.  However, strangely, unexpectedly,  it turned out that the big 3 all used the same syntax.  So, this will just be a quick overview of the differences between SAS and MySQL / SQL Server using INSERT.

Here’s some SAS code that creates a table, inserts some data and then updates it.

proc sql;
CREATE TABLE work.myjunk
(
	name CHAR(1),
	number INT
);
INSERT INTO work.myjunk VALUES ('A',1)
                        VALUES ('B',2)
                        VALUES ('C',3);
UPDATE work.myjunk
SET number =
CASE WHEN name = 'A' then 11
     WHEN name = 'B' then 22
	 ELSE 33
	 END;
SELECT * FROM work.myjunk;
quit;

name    number
--------------
A           11
B           22
C           33

There are two things worth noting in this code. First the INT data type. SAS accepts it, 9 others according to the Study Guide which it will then convert to one of the two SAS data types: number and character. In this case, INT is converted to a number.

I, finally, got smart and created a work database for MySQL and work schema for SQL Server.  It should help my code look a bit more consistent, even if work. means one thing in SAS, another in MySQL and still another in SQL Server.

The other thing, which is what led me to write this post, is that SAS uses a VALUES clause before each row in an INSERT. As you’ll see in the MySQL code (it’s the same with SQL Server) we only use a single VALUES clause.

mysql> CREATE TABLE work.myjunk
    -> (
    ->  name char(1),
    ->  number int
    -> );
Query OK, 0 rows affected (0.06 sec)

mysql> INSERT INTO work.myjunk VALUES ('A',1),('B',2),('C',3);
Query OK, 3 rows affected (0.02 sec)
Records: 3  Duplicates: 0  Warnings: 0

mysql> UPDATE work.myjunk
    -> SET number =
    -> CASE WHEN name = 'A' then 11
    ->      WHEN name = 'B' then 22
    ->   ELSE 33
    ->   END;
Query OK, 3 rows affected (0.02 sec)
Rows matched: 3  Changed: 3  Warnings: 0

mysql> SELECT * FROM work.myjunk;
+------+--------+
| name | number |
+------+--------+
| A    |     11 |
| B    |     22 |
| C    |     33 |
+------+--------+
3 rows in set (0.00 sec)

Hopefully, this makes sense.  You use a single VALUES clause for SQL Server / MySQL and a VALUES clause for each row with SAS.  And, just be aware that you can use a CASE statement in an UPDATE.

24 May

INSERT with a SET

This is something I did not know you could do, but you can use SET for an INSERT statement.  I guess I’m a noob.  Anyways, here is how it works in the three SQL variants that I write about.

SQL Server

INSERT INTO myjunk1
SET id = 1;

Msg 156, Level 15, State 1, Line 2
Incorrect syntax near the keyword 'set'.

I think it’s time I deleted the tables I’ve named junk. Anyways, no surprise that SQL Server doesn’t like it.

MySQL

mysql> INSERT INTO myjunk
    -> SET name = 'E',
    -> number = 6;
Query OK, 1 row affected (0.04 sec)

Hmmm, another junk. Anyways, it works fine here. I’ll give more detail in the next section as I’m really writing this article for SAS.

SAS

proc sql;
CREATE TABLE work.myjunk
(
	id num
);
INSERT INTO myjunk
SET id = 2;
SELECT * 
FROM work.myjunk;
quit;

      id
--------
       2

You can also add multiple values to the table as follows:

proc sql;
CREATE TABLE work.myjunk
(
	id num
);
INSERT INTO myjunk
SET id = 3
SET id = 4
SET id = 5;
SELECT * 
FROM work.myjunk;
quit;

      id
--------
       3
       4
       5

Interestingly, you can recreate a table, albeit this is a temporary table, by running a CREATE TABLE on top of it, which is why there are only 3 values in this table. If you have multiple columns in the table, you treat them exactly like you would an UPDATE statement.

proc sql;
CREATE TABLE work.myjunk
(
	id num,
	id1 num
);
INSERT INTO myjunk
SET id = 3,
   id1 = 13
SET id = 4,
   id1 = 14
SET id = 5,
   id1 = 15;
SELECT * 
FROM work.myjunk;
quit;

      id       id1
------------------
       3        13
       4        14
       5        15

Anyways, that’s pretty much it for INSERT / SET. It’ll probably come up on the SAS Advanced test, assuming I get there, and I have to admit that I’d have blown this one.

22 May

SET Operators: Intersect

INTERSECT, is essentially the reverse of EXCEPT. What it does is return unique records that match in the two tables that intersect. For example,

SELECT name, sex, age
FROM junk
INTERSECT
SELECT name, sex, age
FROM allMale

name	        sex	age
Cameron, L	M	47
Derber, B	M	25
Jones, M	M	29
King, E	        M	35
LaMance, K	M	51
Murray, W	M	27
Peterson, V	M	30
Pitts, D	M	34
Underwood, K	M	60
Warren, C	M	54

In reality it’s simply an inner join. In fact, that’s one way we would write this query in MySQL.

mysql> SELECT j.name, j.sex, j.age
    -> FROM junk j
    -> INNER JOIN allMale a
    -> USING (name, sex, age);
+--------------+------+------+
| name         | sex  | age  |
+--------------+------+------+
| Murray, W    | M    |   27 |
| LaMance, K   | M    |   51 |
| Jones, M     | M    |   29 |
| King, E      | M    |   35 |
| Pitts, D     | M    |   34 |
| Peterson, V  | M    |   30 |
| Cameron, L   | M    |   47 |
| Underwood, K | M    |   60 |
| Derber, B    | M    |   25 |
| Warren, C    | M    |   54 |
+--------------+------+------+
10 rows in set (0.00 sec)

That can also be written a couple of other ways.

mysql> SELECT j.name, j.sex, j.age
    -> FROM junk j, allMale a
    -> WHERE j.name = a.name AND j.sex = a.sex AND j.age = a.age;
+--------------+------+------+
| name         | sex  | age  |
+--------------+------+------+
| Murray, W    | M    |   27 |
| LaMance, K   | M    |   51 |
| Jones, M     | M    |   29 |
| King, E      | M    |   35 |
| Pitts, D     | M    |   34 |
| Peterson, V  | M    |   30 |
| Cameron, L   | M    |   47 |
| Underwood, K | M    |   60 |
| Derber, B    | M    |   25 |
| Warren, C    | M    |   54 |
+--------------+------+------+
10 rows in set (0.00 sec)

Or,

mysql> SELECT DISTINCT j.name, j.sex, j.age
    -> FROM junk j
    -> WHERE EXISTS (SELECT name FROM allMale WHERE j.name = name AND j.sex = sex 
       and j.age = age);
+--------------+------+------+
| name         | sex  | age  |
+--------------+------+------+
| Murray, W    | M    |   27 |
| LaMance, K   | M    |   51 |
| Jones, M     | M    |   29 |
| King, E      | M    |   35 |
| Pitts, D     | M    |   34 |
| Peterson, V  | M    |   30 |
| Cameron, L   | M    |   47 |
| Underwood, K | M    |   60 |
| Derber, B    | M    |   25 |
| Warren, C    | M    |   54 |
+--------------+------+------+
10 rows in set (0.00 sec)

One other note, because SAS, always is just so cute. You can write an INTERSECT with SAS as follows:

proc sql;
SELECT name, age
FROM mysql.admit
WHERE sex = 'M'
INTERSECT ALL
SELECT name, age
FROM mysql.admit;
quit;

Name                 Age
------------------------
Cameron, L            47
Derber, B             25
Jones, M              29
King, E               35
LaMance, K            51
Murray, W             27
Peterson, V           30
Pitts, D              34
Underwood, K          60
Warren, C             54

“We’re developing a new citizenry. One that will be very selective about cereals and automobiles, but won’t be able to think.”
Rod Serling

SAS is particularly vague about what ALL does in the case of INTERSECT but it’s essentially the same as UNION and UNION ALL. In other words, it returns all records that match both tables in the INTERSECT rather than only unique records which is what a simple INTERSECT would do.

SQL Server does not support INTERSECT ALL and MySQL does not support INTERSECT at all.

I hope this is clear to anyone reading this. I always found SET operators really clean. Unfortunately, we’re about to enter the Twilight Zone as SAS has one that is unique to them, but then, that’s “situation normal” for SAS.

20 May

SET Operators: EXCEPT

I was thinking this would be hard going into it, but, it’s actually really easy, except, that EXCEPT doesn’t exist in SQL Server.

EXCEPT is very simple.  What it does is return every unique row in the first result set, that is not in the second result set.  Here are two very simple examples:

SELECT 1 AS r1,2 AS r2,3 AS r3
EXCEPT
SELECT 3,4,5;

r1	r2	r3
1	2	3

SELECT 1 AS r1,2 AS r2,3 AS r3
EXCEPT
SELECT 1,2,3;

r1	r2	r3

Those are simple queries but they show how the EXCEPT operator works. Now, what happens if we have records that are duplicates in the first table? Well, this happens:

SELECT 1 AS r1,2 AS r2,3 AS r3
UNION ALL
SELECT 1,2,3
r1	r2	r3
1	2	3
1	2	3

(
SELECT 1 AS r1,2 AS r2,3 AS r3
UNION ALL
SELECT 1,2,3
)
EXCEPT
SELECT 3,4,5

r1	r2	r3
1	2	3

That’s really clean compared to most of my posts, and, it’s going to stay that way. The reason? I’m not going to mess with SAS and I’m going to let MySQL go right ahead not doing it. If you decide to do it in MySQL, and one of these days, I’ll do it, you need to get a bit creative with a JOIN or a couple of other options such as a correlated subquery and NOT EXISTS.

OK, I talked myself into it, but only because I literally just did this the other day.

First, I’ll create a new table that has just the records where sex = ‘M’

mysql> CREATE TABLE allMale AS SELECT * FROM junk WHERE sex = 'M';
Query OK, 10 rows affected (0.11 sec)
Records: 10  Duplicates: 0  Warnings: 0

mysql> SELECT name, sex, age FROM allMale;
+--------------+------+------+
| name         | sex  | age  |
+--------------+------+------+
| Murray, W    | M    |   27 |
| LaMance, K   | M    |   51 |
| Jones, M     | M    |   29 |
| King, E      | M    |   35 |
| Pitts, D     | M    |   34 |
| Peterson, V  | M    |   30 |
| Cameron, L   | M    |   47 |
| Underwood, K | M    |   60 |
| Derber, B    | M    |   25 |
| Warren, C    | M    |   54 |
+--------------+------+------+
10 rows in set (0.00 sec)

Maybe I shouldn’t have called these tables junk?

Anyways, here is a query in SQL Server using the EXCEPT operator.

SELECT name, sex, age
FROM junk
EXCEPT
SELECT name, sex, age
FROM allMale

name	        sex	age
Almers, C	F	34
Bonaventure, T	F	31
Eberhardt, S	F	49
Ivan, H	        F	22
Johnson, R	F	43
Nunnelly, A	F	44
Oberon, M	F	28
Quigley, M	F	40
Reberson, P	F	32
Takahashi, Y	F	43
Wilcox, E	F	41

Now, for the MySQL version.

mysql> SELECT DISTINCT name, sex, age
    -> FROM JUNK j
    -> WHERE NOT EXISTS (SELECT name FROM allMale WHERE j.name = name AND j.
age = age AND j.weight = weight);
+----------------+------+------+
| name           | sex  | age  |
+----------------+------+------+
| Almers, C      | F    |   34 |
| Bonaventure, T | F    |   31 |
| Johnson, R     | F    |   43 |
| Reberson, P    | F    |   32 |
| Eberhardt, S   | F    |   49 |
| Nunnelly, A    | F    |   44 |
| Oberon, M      | F    |   28 |
| Quigley, M     | F    |   40 |
| Takahashi, Y   | F    |   43 |
| Ivan, H        | F    |   22 |
| Wilcox, E      | F    |   41 |
+----------------+------+------+
11 rows in set (0.00 sec)

I had to write that about 10 times because I kept forgetting to change the SELECT clause. Yeah, for me, SELECT = ‘HARD’, EXISTS = ‘Easy’.

The only part of this query that really matters is the WHERE clause. Essentially it uses a correlated subquery to compare the two data sets and where the record NOT EXISTS in allMale, it is returned to the result set. It’s not very clean, and all things being equal, I’d rather use EXCEPT.

OK, the SELECT matters too.  I have to use the DISTINCT there so the query won’t return duplicate values, or, as the documentation I’ve read for EXCEPT says “unique values only”.

And just for kicks, Oracle doesn’t have an EXCEPT operator either. Over on that part of the ranch, they roll with MINUS.

Fortunately, I’m done with EXCEPT, except, until I do the “SAS is special” operator article.

19 May

SET Operators: UNION

The easiest set operator to deal with is a UNION.  It’s basically a data set appended to a data set.  It has a couple of minor quirks, column naming, ordering, and then, of course, SAS picked the path least chosen, but for the most part, anytime you see a UNION, or a UNION ALL, it’s going to work the same way.

The code for this example, except where noted, is in MySQL.

Here is a very simple example of a UNION.

mysql> SELECT 1,2,3
    -> UNION
    -> SELECT 4,5,6;
+---+---+---+
| 1 | 2 | 3 |
+---+---+---+
| 1 | 2 | 3 |
| 4 | 5 | 6 |
+---+---+---+
2 rows in set (0.00 sec)

As you can see the second SELECT is added to the first set. Typically, the data types do not have to match, as automatic type conversion will take place. Except, of course, if you are working with SAS, where the following will happen:

proc sql;
SELECT name, age
FROM mysql.admit
UNION
SELECT age, name
FROM mysql.admit;
quit;

ERROR: Column 1 from the first contributor of UNION is not the same type as its counterpart from the second.
ERROR: Column 2 from the first contributor of UNION is not the same type as its counterpart from the second.

A straight UNION will remove duplicates similar to what a DISTINCT does in a SELECT statement.

mysql> SELECT 1,2,3
    -> UNION
    -> SELECT 1,2,3
    -> UNION
    -> SELECT 3,4,5;
+---+---+---+
| 1 | 2 | 3 |
+---+---+---+
| 1 | 2 | 3 |
| 3 | 4 | 5 |
+---+---+---+
2 rows in set (0.00 sec)

If you, instead, write the query as a UNION ALL it will include all rows, including any duplicates. This applies in SAS, although, they approach the ALL keyword slightly differently but I’ll deal with that in a later article.

mysql> SELECT 1,2,3
    -> UNION ALL
    -> SELECT 1,2,3
    -> UNION ALL
    -> SELECT 1,2,3;
+---+---+---+
| 1 | 2 | 3 |
+---+---+---+
| 1 | 2 | 3 |
| 1 | 2 | 3 |
| 1 | 2 | 3 |
+---+---+---+
3 rows in set (0.00 sec)

You can also mix a UNION and a UNION ALL.

mysql> SELECT 1,2,3
    -> UNION ALL
    -> SELECT 1,2,3
    -> UNION
    -> SELECT 1,2,3;
+---+---+---+
| 1 | 2 | 3 |
+---+---+---+
| 1 | 2 | 3 |
+---+---+---+
1 row in set (0.00 sec)

Each UNION only applies to the SELECT statements that it separates.

You can also alias a UNION, however, the alias must be applied to the first SELECT statement’s columns. You cannot apply it to later columns, well, you can, but as you can see it won’t work. For example:

mysql> SELECT 1 as one, 2 as two, 3 as three
    -> UNION
    -> Select 4,5,6;
+-----+-----+-------+
| one | two | three |
+-----+-----+-------+
|   1 |   2 |     3 |
|   4 |   5 |     6 |
+-----+-----+-------+
2 rows in set (0.00 sec)

mysql> SELECT 1,2,3
    -> UNION ALL
    -> SELECT 1 as one, 2 as two, 3 as three;
+---+---+---+
| 1 | 2 | 3 |
+---+---+---+
| 1 | 2 | 3 |
| 1 | 2 | 3 |
+---+---+---+
2 rows in set (0.00 sec)

ORDER BY statements must be applied at the end of the UNION. For example:

mysql> SELECT name FROM junk ORDER BY name
    -> UNION
    -> SELECT name FROM junk;
ERROR 1221 (HY000): Incorrect usage of UNION and ORDER BY
mysql>

mysql> SELECT name FROM junk
    -> UNION
    -> SELECT name FROM junk ORDER BY name;
+----------------+
| name           |
+----------------+
| Almers, C      |
| Bonaventure, T |
| Cameron, L     |
| Derber, B      |
| Eberhardt, S   |
| Ivan, H        |
| Johnson, R     |
| Jones, M       |
| King, E        |
| LaMance, K     |
| Murray, W      |
| Nunnelly, A    |
| Oberon, M      |
| Peterson, V    |
| Pitts, D       |
| Quigley, M     |
| Reberson, P    |
| Takahashi, Y   |
| Underwood, K   |
| Warren, C      |
| Wilcox, E      |
+----------------+
21 rows in set (0.04 sec)

And now, it gets a bit funky, because SAS joined the party.

This is viable in SAS:

proc sql;
SELECT name, age
FROM mysql.admit
UNION
SELECT name
FROM mysql.admit;
quit;

Name                 Age
------------------------
Almers, C              .
Almers, C             34
Bonaventure, T         .
Bonaventure, T        31
Cameron, L             .
Cameron, L            47
Derber, B              .
Derber, B             25
Eberhardt, S           .
Eberhardt, S          49
Ivan, H                .
Ivan, H               22
Johnson, R             .
Johnson, R            43

MySQL and SQL Server will both error.

mysql> SELECT 1,2
-> UNION
-> SELECT 1,2,3;
ERROR 1222 (21000): The used SELECT statements have a different number of column

In other words, SQL Server, and MySQL, expect that a UNION operator will have the same number of columns.  SAS, doesn’t care, and simply NULLs, or in SAS terminology, sets those fields to missing.

SET operators, all things considered, are really easy to use, assuming SAS doesn’t get involved and even there, as long as you stay with a UNION or UNION ALL, it’s sort of, maybe, kind of, close.

As far as the SAS Advanced test goes, I think I’ll keep in mind that SAS allows a different number of columns, and, it doesn’t allow different data types in a column. It also has other quirks but I’ll deal with those in later posts.

18 May

GROUP BY, Again, This Time It’s Final, No Really It Is.

I think this will be the last of these.  I’ve gotten four posts out of GROUP BY and this time I’m going to double-dip, with the HAVING clause.  So, lets start with SQL Server and blow things up.

SQL Server

SELECT actlevel AL, SUM(height) AS 'sumHeight'
FROM junk
GROUP BY AL
HAVING SUM(height) > 400;

Msg 207, Level 16, State 1, Line 3
Invalid column name 'AL'.

SELECT actlevel AL, SUM(height) AS 'sumHeight'
FROM junk
GROUP BY actlevel
HAVING sumHeight > 400;

Msg 207, Level 16, State 1, Line 4
Invalid column name 'sumHeight'.

Both of these errors are saying that you can’t use an alias in a GROUP BY or HAVING clause. Honestly, this is kind of the expected behavior if you understand how SQL theoretically works (another post).

However, the other two guys I write about are just fine with it.

MySQL and SAS

mysql> SELECT actlevel AL, sum(height) as mysum
    -> FROM junk
    -> GROUP BY AL
    -> HAVING mysum > 400;
+------+-------+
| AL   | mysum |
+------+-------+
| HIGH |   491 |
| LOW  |   465 |
| MOD  |   477 |
+------+-------+
3 rows in set (0.00 sec)

proc sql;
SELECT actlevel AL, sum(height) AS mysum
FROM mysql.admit
GROUP BY AL
HAVING mysum > 400;
quit;

ERROR 22-322: Syntax error, expecting one of the following: !, !!, &, (, *, **, +, ',', -, '.', 
/, <, <=, <>, =, >, >=, ?, AND, AS, CONTAINS, EQ, EQT, FROM, GE, GET, GT, GTT, LE, LET, LIKE, LT, 
LTT, NE, NET, OR, ^=, |, ||, ~=.

Ah, SAS, the 4th grade comedian that drove the teacher crazy. SAS requires the AS keyword.

proc sql;
SELECT actlevel AS AL, sum(height) as mysum
FROM mysql.admit
GROUP BY AL
HAVING mysum > 400;
quit;

AL       mysum
--------------
HIGH       491
LOW        465
MOD        477

So, that’s all good. I can end now, right. I mean, I wouldn’t be tempted to do something exotic, just to see what SAS, or maybe even MySQL, does with it, would I? Nope, not me.

mysql> SELECT actlevel 'AL', sum(height) as mysum
    -> FROM junk
    -> GROUP BY 'AL'
    -> HAVING mysum > 400;
+------+-------+
| AL   | mysum |
+------+-------+
| HIGH |  1433 |
+------+-------+
1 row in set (0.00 sec)

Err, OK. Maybe this will be better.

mysql> SELECT actlevel AL, sum(height) as 'mysum'
    -> FROM junk
    -> GROUP BY AL
    -> HAVING 'mysum' > 400;
Empty set, 1 warning (0.00 sec)

Oops! MySQL, ever the professionals at making it clear what went wrong. But, I wonder how SAS will handle this code.

proc sql;
SELECT actlevel 'AL', sum(height) as mysum
FROM mysql.admit
GROUP BY 'AL'
HAVING mysum > 400;
quit;

AL       mysum
--------------
HIGH      1433
HIGH      1433
LOW       1433
MOD       1433
LOW       1433
HIGH      1433
MOD       1433
MOD       1433
LOW       1433
LOW       1433
HIGH      1433
LOW       1433
MOD       1433
HIGH      1433
MOD       1433
LOW       1433
MOD       1433
HIGH      1433
LOW       1433
HIGH      1433
MOD       1433
Okey, dokey.

And,
proc sql;
SELECT actlevel AL, sum(height) as 'mysum'
FROM junk
GROUP BY AL
HAVING 'mysum' > 400;
quit;

ERROR 22-322: Syntax error, expecting one of the following: !, !!, &, (, *, **, +, ',', -, 
'.', /, <, <=, <>, =, >, >=, ?, AND, AS, CONTAINS, EQ, EQT, FROM, GE, GET, GT, GTT, LE, LET, 
LIKE, LT, LTT, NE, NET, OR, ^=, |, ||, ~=.

Lets try this, instead:

proc sql;
SELECT actlevel AL, sum(height) as 'mybum'
FROM junk
GROUP BY AL
HAVING 'mybum' > 400;
quit;

ERROR: Expression using greater than (>) has components that are of different data types.

Really? No, Really? SAS is just a giant box of endless posts.

Well, I guess it has to be this way.

proc sql;
SELECT actlevel AS AL, sum(height) AS mybum
FROM mysql.admit
GROUP BY AL
HAVING mybum > 400;
quit;

AL       mysum
--------------
HIGH       491
LOW        465
MOD        477

Or, I suppose, you could just write it like you are supposed to.

proc sql;
SELECT actlevel AS AL, sum(height) AS mybum
FROM mysql.admit
GROUP BY actlevel
HAVING sum(height) > 400;
quit;

AL       mysum
--------------
HIGH       491
LOW        465
MOD        477

I just wish I could trust SAS to write the SAS Advanced certification that cleanly but after the Base I simply don’t trust them not to use all of this stuff.

I’m out!