SIS password hashes

docs/blackboard/sis/sis-password-hashes.md

@@ -0,0 +1,256 @@
+---
+title: "SIS Password Hash Security"
+sidebar_position: 2
+id: sis-password-hashes
+author: John Z. Walthall
+published: ""
+edited: ""
+---
+
+## Introduction
+
+You can create or set user passwords by SIS using the 'Person' data type. There are three ways to do this:
+
+1. Plaintext (not recommended)
+2. MD5 (not recommended)
+3. Salted SHA-1 ("SSHA", recommended)
+
+Here is an example of all three formats. (In each case: the password is 'cyan')
+
+```
+user_id|external_person_key|lastname|firstname|passwd|pwencryptiontype|data_source_key
+jshaw|jshaw|Shaw|James|{SSHA}foV2dGZ/2FLNdmJUNEpXZ8ijfiGAriwuB9AYrQ==|SSHA|exterminate
+jplain|jplain|Plain|Jane|cyan||exterminate
+md5|md5|Five|Maddy|6411532ba4971f378391776a9db629d3|MD5|exterminate
+```
+
+## Supplying a Password Hash
+
+These hash formats are not the one used by the LMS. When a password is set in the GUI it is hashed using a
+[512-bit SHA-2-based](https://help.blackboard.com/Learn/Administrator/SaaS/Security/Identification_Authentication) hash.
+
+If you supply the field `passwd`, but not the field `pwencryptiontype` (or: set this field to blank) the input will be
+hashed by the LMS using the aforementioned hash.
+
+Otherwise, the input will be copied verbatim into the database. When the user logs in: which is the only time the clear-
+text of the password is accessible to Blackboard, the cleartext will be re-hashed using a strong hash and the existing
+one overwritten.
+
+If you set 'change on update' for the password field: the stronger hash will be overwritten.
+
+### Using MD5
+
+:::danger
+It's not recommended to use MD5. MD5 is obsolete and no longer secure. It may be subject to removal in a future version
+of Blackboard Learn.
+:::
+
+The MD5 hash is trivial: it's simply the standard hex representation of the MD5sum of the string. So if the password is
+'cyan' then
+
+```shell
+$ echo -n "cyan" | md5
+6411532ba4971f378391776a9db629d3
+```
+
+### Using SSHA
+
+This format is an informal standard and our implementation is _similar to_ but _not the same as_ [the format used by
+`slappasswd(8)`](https://github.com/openldap/openldap/blob/34813d9cba02a74216a784636a8d5f0f986d73c7/libraries/liblutil/passwd.c#L749-L779)
+from the OpenLDAP project. The key difference is the salt-size. Blackboard Learn uses an **8-byte** salt. OpenLDAP uses
+a **4-byte** salt. Since the salt-size is not stored in the format, hashes with different salt-lengths are not
+compatible.
+Therefore, the hashes generated by `slappasswd(8)` cannot be used.
+
+### Algorithm Description
+
+1. For each password: create 8 random bytes of 'salt'. Never re-use salts. Use a Cryptographically Secure Psuedo-Random
+   Number Generator, or a true environmentally sourced random-number generator.
+2. Convert the password string to a byte array using UTF-8.
+3. Digest the concatenation of the password bytes + the salt bytes as SHA-1. The SHA-1 digest must be a byte array
+   itself, if the implementation produces a hex string, it must be converted to a byte array by parsing each pair of
+   characters as an unsigned hexadecimal byte.
+4. Append another copy of the same salt bytes.
+5. Encode this as Base64 in ASCII. Use the default dictionary: not the 'url-safe' one.
+7. Prefix with `{SSHA}`.
+
+### Step by Step
+
+1. Let the password be `nucleus`
+2. Let the salt be `[21, F0, 25, 09, 15, D2, 68, 1F]` (interpreted as an array of unsigned bytes.) Remember: it must
+   always be eight new random bytes.
+3. The UTF-8 bytes of 'nucleus' are `[6E, 75, 63, 6C, 65, 75, 73]`
+4. Thus the value to be digested is `[6E, 75, 63, 6C, 65, 75, 73, 21, F0, 25, 09, 15, D2, 68, 1F]`
+5. The SHA-1 digest of this (interpreted as an array of unsigned bytes) is
+   `[90, FC, 6D, A2, C9, EA, 04, 10, 83, 20, C4, AC, 15, 73, A7, 49, BD, 88, 7A, 63]`
+6. Appending the salt to that is
+   `[90, FC, 6D, A2, C9, EA, 04, 10, 83, 20, C4, AC, 15, 73, A7, 49, BD, 88, 7A, 63, 21, F0, 25, 09, 15, D2, 68, 1F]`
+7. The Base64 encoding of this, with prefix is `{SSHA}kPxtosnqBBCDIMSsFXOnSb2IemMh8CUJFdJoHw==`
+
+### Code Examples
+
+Here are some code examples.
+
+:::caution
+A particular source of confusion, and many examples on the Internet stumble on this confusion, is that the SHA-1 output
+must be bytes, not a hex string. If the programming language or library you choose returns a hex string for SHA-1, then
+it must be recast as a byte array. That is to say: `EF` must be a single byte: `239` (unsigned), not a pair of bytes
+equal to unsigned `69` (the '`E`') and `70` (the '`F`'.)
+:::
+
+:::info
+According to the SHA-1 specification: an empty string can be hashed. You must not do this. In the SIS API, an empty
+password indicates the LMS should generate a random password. Setting the password to the SSHA hash of empty string
+results in a blank password being set.
+:::
+
+#### Java
+
+```java
+import java.nio.charset.StandardCharsets;
+import java.security.MessageDigest;
+import java.security.NoSuchAlgorithmException;
+import java.security.SecureRandom;
+import java.util.Arrays;
+import java.util.Base64;
+
+/**
+ * Encoder for our nonstandard SSHA variant using 8-byte salt.
+ */
+public final class VariantSSHA {
+
+    /**
+     * Encode string to the hash
+     * @param password The password to encode
+     * @return the variant-SSHA hash.
+     */
+    public static String variantSSHAEncode(String password) {
+        if (password.isBlank()) {
+            // empty-string password has a special meaning in Learn. But a hash of empty-string is a valid hash
+            // bail out to preserve semantics
+            throw new IllegalArgumentException("Password cannot be blank");
+        }
+        // The salt is always 8 bytes. This is incompatible with slappasswd(8)
+        byte[] salt = new byte[8];
+        SecureRandom rand = null;
+        try {
+            rand = SecureRandom.getInstanceStrong();
+        } catch (NoSuchAlgorithmException e) {
+            throw new RuntimeException("FATAL: can't load secure randomness", e);
+        }
+        rand.nextBytes(salt);
+
+        byte[] bytesUTF8 = password.getBytes(StandardCharsets.UTF_8);
+
+        try {
+            // hash the combination of the password + the salt.
+            MessageDigest sha1Digest = MessageDigest.getInstance("SHA1");
+            sha1Digest.update(bytesUTF8);
+            sha1Digest.update(salt);
+            byte[] binaryHash = sha1Digest.digest();
+
+            // append the salt to the hash
+            byte[] hashPlusSalt = new byte[binaryHash.length + salt.length];
+            System.arraycopy(binaryHash, 0, hashPlusSalt, 0, binaryHash.length);
+            System.arraycopy(salt, 0, hashPlusSalt, binaryHash.length, salt.length);
+
+            // This is mostly security theater; but customary
+            Arrays.fill(salt, (byte) 0);
+            Arrays.fill(bytesUTF8, (byte) 0);
+
+            String stringHash = "{SSHA}" + Base64.getEncoder().encodeToString(hashPlusSalt);
+            Arrays.fill(hashPlusSalt, (byte) 0);
+
+            return stringHash;
+        } catch (NoSuchAlgorithmException e) {
+            // can't happen. JSSE requires all implementing runtimes to support SHA-1
+            throw new RuntimeException(e);
+        }
+    }
+
+    /**
+     * Convenience method to use as a simple utility. Output format is {@code original_password + tab + SSHA}
+     *
+     * <pre>
+     * $ java VariantSSHA.java the quick brown fox" "jumps over the lazy dog" "when zombies arrive" "quickly fax judge patty"
+     *     the quick brown fox	{SSHA}r+QLZ86dFWWp0oXhGC3nW5U/p08DvFVyKH1M/w==
+     * jumps over the lazy dog	{SSHA}yxUScjSM42EBpL2qB7I2wLf/CLHBQX0No18z/w==
+     *     when zombies arrive	{SSHA}Yvot6sr1F7XNahlwY0KeXmmukpw19oYSJnZhRQ==
+     * quickly fax judge patty	{SSHA}f01o7IJGet6TzvizERwuVzPX7Ud09Pu3HGJeZg==
+     * </pre>
+     * @param args strings to encode.
+     */
+    static void main(String... args) {
+        if (args.length == 0) {
+            System.err.println("Try again");
+            System.err.println("Usage: java VariantSSHA password1 password2 password3...");
+        } else {
+            int pad = Arrays.stream(args).mapToInt(String::length).max().orElse(10);
+            for (String password : args) {
+                System.out.printf("%" + pad + "s\t%s%n", password, variantSSHAEncode(password));
+            }
+        }
+    }
+}
+```
+
+#### Python
+
+```python
+#!/usr/bin/env python3
+"""
+Generate Blackboard's nonstandard 8-byte-salted SSHA variant. 
+"""
+import base64
+import hashlib
+import os
+import sys
+
+
+def variant_ssha_encode(password: str):
+    if password == "":
+        raise RuntimeError("Password can not be empty")
+
+    # Generate 8 random bytes of salt. os.urandom is cryptographically secure
+    salt = os.urandom(8)
+
+    # convert the password to bytes
+    bytes_utf8 = password.encode("utf-8")
+
+    sha1digest = hashlib.sha1()
+
+    # equivalent to byte_utf8 + salt
+    sha1digest.update(bytes_utf8)
+    sha1digest.update(salt)
+
+    # append the salt to the end of the SHA-1 digest (note the digest must be bytes, not hex string)
+    hash_with_salt = sha1digest.digest() + salt
+
+    # encode to base64, pre-fix the identifier and return.
+    return "{SSHA}" + base64.b64encode(hash_with_salt).decode("ascii")
+
+
+def main():
+    """
+    Convenience method to use as a simple utility
+    invocation:
+        $ python3 generate_ssha_hash.py "the quick brown fox" "jumps over the lazy dog" "when zombies arrive" "quickly fax judge patty"
+            the quick brown fox	{SSHA}Ffy5dpkMeMIiebd+Sqtu0FJOV6xdAh4Wp9aeSA==
+        jumps over the lazy dog	{SSHA}SmYwGocJidrBS9AfBid9P/JUUOxhTZLylWcKQw==
+            when zombies arrive	{SSHA}layQWCu+uVrFmXeKE4ZeqPGzCJ87OVI0zAnjJQ==
+        quickly fax judge patty	{SSHA}IJbtvQYh6TocBq5m4yoU0sVRvUdMrR+hZUHxCQ==
+    """
+    if len(sys.argv) == 1:
+        print("Try again\nUsage: python3 generate_ssha_hash.py password1 password2 password3...")
+        sys.exit(1)
+    passwords = sys.argv[1:]
+    pad = len(max(passwords, key=len))
+    for pw in passwords:
+        ssha = variant_ssha_encode(pw)
+        padded = pw.rjust(pad)
+        print(f"{padded}\t{ssha}")
+
+
+if __name__ == "__main__":
+    main()
+```

docs/blackboard/sis/welcome.md

@@ -0,0 +1,36 @@
+---
+title: "Getting started with SIS"
+sidebar_position: 1
+id: getting-started
+author: John Z. Walthall
+published: ""
+edited: ""
+---
+
+The SIS (Student Information Systems) API is a classic API oriented around bulk or event-driven data loading from a SIS
+or ERP program like Anthology Student, Banner or Colleague. Most aspects of SIS are [documented on our main help site](https://help.blackboard.com/Learn/Administrator/SaaS/Integrations/Student_Information_System),
+but some developer documentation is provided here.
+
+## SIS Types
+
+The "prime" data type is "[Snapshot Flat File](https://help.blackboard.com/Learn/Administrator/SaaS/Integrations/Student_Information_System/SIS_Integration_Types/Snapshot_Flat_File)", which is a bulk-loading oriented delimited text file. This is the 'native'
+format of Blackboard and has the widest support for the LMS's many data types.
+
+We also support three industry-standard formats:
+
+* [LIS 2.0 Final and Draft](https://help.blackboard.com/Learn/Administrator/SaaS/Integrations/Student_Information_System/SIS_Integration_Types/LIS)
+* [IMS Enterprise 1.1](https://help.blackboard.com/Learn/Administrator/SaaS/Integrations/Student_Information_System/SIS_Integration_Types/Enterprise_1.1)
+* [IMS Enterprise 1.1 - Vista](https://help.blackboard.com/Learn/Administrator/SaaS/Integrations/Student_Information_System/SIS_Integration_Types/Enterprise_1.1) 
+
+These are usually used in an event-driven workflow.
+
+## Security
+
+SIS activity occurs at a high level of privilege. It can create, remove, or delete anything. A user that gains access to
+an SIS username and password could send arbitrary data files that cause arbitrary changes. 
+
+SIS data files also contain potentially sensitive data and access to them should be strictly controlled. (Note: Support
+may require the feed file for problem investigation.) 
+
+One particular case is setting passwords by SIS. They can be supplied in cleartext, but this is not recommended and they
+should be provided in [hashed form.](sis-password-hashes.md)

sidebar.js

@@ -209,6 +209,12 @@ const sidebars = {
             },
           ],
         },
+        {
+            SIS: [
+                "blackboard/sis/getting-started",
+                "blackboard/sis/sis-password-hashes"
+               ]
+        },
       ],
     },
     // Student