This week I worked on my test matrix and got the basic plan approved. I also worked on the Python script to transfer information from the log file to Excel. As well as, duplicating datasets, spraying additional datasets on the cluster, and running kubectl logs to obtain the summary values for different number of records.
Log File to Excel
I worked on coming up with a python script to transfer the log data from my log file into an excel for better readbility. You can view the script here. The image below displays the output of the python file.
Final Test Matrix
After getting feedback from my mentors, I came up with a basic plan for my test matrix.
Spraying & Duplicating Datasets
I learned how to duplicate my datasets to increase the record number as well as importing new data files into the ECL Watch.
To spray the data on the ECL Watch you have to open ECL Watch in your browser using localhost:8010, and then upload the csv file to the landing zone, spray the data using the delimited option and then create an .ecl code to define a RECORD structure and a DATASET definition for the dataset. You can also follow this video to follow along if this is unclear.
To duplicate dataset, simple add the same dataset multiple times to increase the number of records.
rec:=record
string s;
end;
ds:=dataset([{‘a’},{‘b’},{‘c’}],rec);
ds2:=ds+ds;
ds2;
nivedhahpccblogpost 