madisonjmyers / using-the-apache-systemml-api-on-a-spark-shell- Goto Github PK

@MadisonJMyers In general, this is great! It gets the user using Spark, SystemML, and IAE. Here are some thoughts as I went through it:

• It's subject to preference, but I would lowercase "Shell", and slightly alter the wording from "on a Spark Shell" to "in a Spark shell".
• I might alter the wording of "You can log into Spark Shell, load SystemML on the shell, ..." to "You start a Spark shell, import SystemML, ....".
• I would remove the assumptions for SystemML section with the brew install commands since IAE will already have Spark installed.
• I'm not sure opening the Ambari webpage necessary. I think it should be fine to ssh in directly.
• It's not yet clear where the ssh command is from the readme -- I found it under "Connection Details".
• Under the "Our next step is to parallelize the information, read in two matrices as RDDs, getting the sum of the first, the sum of the second and a message." section, can you remove the scala> prefixes so that the user can simply copy & paste?
• In that same section, there is an issue with the last quotation mark (") in the lines, such as in val data1 = sc.parallelize(Array("1.0,2.0", "3.0,4.0”)) in which that final " has been encoded as ”, and so it doesn't work at the command line. Can you update it to a " mark? There are a few lines in the section with that problem.
• Also, for all of the code sections, can you wrap them up as code blocks (``` marks on the lines before and after each block -> https://guides.github.com/features/mastering-markdown/#examples)? That will probably help with the encoding and formatting issues, and make it easier to copy & paste.
• In the "Now print your script info. You should see:" section, can you split this into the code to run (println(script.info)) and the expected output the user should see? Code blocks will help a lot as well.
• When I run the code in "Execute your script and get your results!", I get an error. I think it is due to the formatting of val s = ... in "Our next step is to parallelize the information, read in two matrices as RDDs, getting the sum of the first, the sum of the second and a message.". The vertical bars (|) shouldn't be present.
• The line val script = dmlFromUrl(scriptUrl).in("A", habermanRDD, habermanMetadata).in("K", typesRDD, typesMetadata).in("$CONSOLE_OUTPUT", true) results in an error due to typesMetadata not being available.
• Feature [1]: Scale ... and the next few sections are bolded by accident.