November 21, 2015

IDR 10: Test data initialization

Yesterday, I was humbly reminded just how much I still *don't* know about R.

First, I learned that there is apparently no straightforward way to map a row from one data.frame into the columns of another. This meant that it would be much simpler for my code to store all training and "test" examples for each experiment in a shared data frame, with a new column indicating training vs. test. I finally managed to rewrite that code and now have a single CSV being dumped with the new column and all examples.

Then, I finally dealt with the fact that all of my data was being loaded with 0s and NAs instead of 0s and 1s. I was initializing a data.frame from a matrix of 0s. This led the data.frame to think that the only "level" in every variable in my model was "0" (the variables were at least correctly being interpreted as categorical variables or factors in R). So when I actually wanted to use a new value, this was essentially rejected, and the value set to "NA".

There are apparently some ways around this, but it was easier for me to rewrite my code to set the 0 and 1 values within the matrix instead. Now creation of the data.frame looks to be working correctly (I still have a 2 hour cycle time to run this on a real dataset).

So this turned out to be another frustrating day by the end, but it did have its moments of hope before these problems crept up. Here's hoping that today is a better one!

PS, I also got that dental work fixed that I mentioned a couple of posts ago. It actually hurt a bit more than the same procedure did last time, but hopefully that will go away.

IDR Series

8 comments:

  1. Thank you so much for sharing this pretty post, it was so good to read and useful to improve my knowledge as
    appvn app

    ReplyDelete
  2. I think I have never watched such online diaries ever that has absolute things with all nuances which I need. So thoughtfully update this ever for us.
    difference between analysis and analytics

    ReplyDelete
  3. I'm cheerful I found this blog! Every now and then, understudies need to psychologically the keys of beneficial artistic articles forming. Your information about this great post can turn into a reason for such individuals.
    data science course in malaysia

    ReplyDelete
  4. You finished certain solid focuses there. I did a pursuit regarding the matter and discovered almost all people will concur with your blog.
    iot course in noida

    ReplyDelete
  5. Superb exertion to make this blog more awesome and appealing.
    digital marketing course malaysia

    ReplyDelete
  6. Stunning! Such an astonishing and supportive post this is. I incredibly love it. It's so acceptable thus wonderful. I am simply astounded.
    hrdf contribution

    ReplyDelete
  7. Hello Sir I saw your blog, It was very nice blog, and your blog content is awesome.Digital Marketing

    ReplyDelete