Unbundle error when the xgboost model is too large

[rvalieris]

Follow up on rstudio/vetiver-r#300 (comment)

When the xgboost object raw vector is too long (>2^31), the unbundle function throws this error:

m <- bundle::unbundle(v$model)
Error in xgboost::xgb.load.raw(object, as_booster = TRUE): long vectors not supported yet: ../../src/include/Rinlinedfuns.h:537
Traceback:

1. unbundle.bundle(v$model)
2. x$situate(get_object(x))
3. bundle::swap_element(object, "fit", "fit")
4. unbundle(component)
5. unbundle.bundle(component)
6. x$situate(get_object(x))
7. bundle::swap_element(object, "fit")
8. unbundle(component)
9. unbundle.bundle(component)
10. x$situate(get_object(x))
11. xgboost::xgb.load.raw(object, as_booster = TRUE)


# original xgboost object
length(xgb.obj$raw)
2356032791

length(xgb.obj$raw) > 2**31
TRUE

In my case this happened with a model that had 4034 nrounds and 30 variables, but I'm not sure what is the best way to reproduce this.

I found related discussion here: dmlc/xgboost#5794
and as a result xgboost devs wrote this doc: https://rdrr.io/cran/xgboost/man/a-compatibility-note-for-saveRDS-save.html

but neither addresses directly the length of raw issue, they are discussing overall issues with saving and loading xgboost models.

in my own tests, I found that xgb.save/xgb.load works because (I think) it reads/writes directly to a file.