Feature selection in mlr3spatial

[walinchus]

The mlr3spatial package requires loading in sample points: https://github.com/mlr-org/mlr3spatial/blob/main/README.md

It says the data is from the built-in toy task Leipzig which comes from the Copernicus Open Access hub. But the hub doesn't download .tif files. (You can download a variety of mapping files.)

Chapter 8.3.6.2 Spatial prediction explains how to use the raster image for prediction, but there's no explanation of how to get point vector data and raster images of the area you want to map.

And there's no other way to do feature selection with the spatial task.

[larskotthoff]

Thanks, we'll take a look -- @pat-s

[pat-s]

It says the data is from the built-in toy task Leipzig which comes from the Copernicus Open Access hub. But the hub doesn't download .tif files. (You can download a variety of mapping files.)

@be-marc added the dataset, he can probably say more or extend the documentation.

Chapter 8.3.6.2 Spatial prediction explains how to use the raster image for prediction, but there's no explanation of how to get point vector data and raster images of the area you want to map.

You need to bring your own point and raster data, similar as you would bring your own CSV data.
In the end, it's just a special format for storing the information used for modeling.

And there's no other way to do feature selection with the spatial task.

I am not sure what you mean here, {mlr3spatial} does not aim to cover FS. See {mlr3fselect} for FS in mlr3.

[be-marc]

It says the data is from the built-in toy task Leipzig which comes from the Copernicus Open Access hub. But the hub doesn't download .tif files. (You can download a variety of mapping files.)

Yes, I created a .tif file from the copernicus open access format (.jp2 and auxiliary files). But that's something you don't do with mlr3spatial. mlr3spatial handles raster objects from the packages terra, raster, and stars. Maybe take a look at these packages. They load spatial data into R. Point vector data is loaded with the sf package.

And there's no other way to do feature selection with the spatial task.

This works

library(mlr3spatial)
library(mlr3fselect)

instance = fselect(
  method = "random_search",
  task =  tsk("leipzig"),
  learner = lrn("classif.rpart"),
  resampling = rsmp("holdout"),
  measure = msr("classif.ce"),
  term_evals = 10,
  batch_size = 5
)

[walinchus]

Thank you this is very helpful.

So let me see if I understand this correctly: the leipzig toy task has point data and other features as well. (I cannot find anywhere what b02, b03, etc are actually measuring but I am assuming something about each point.)

When I run this code, it tells me which features are important:

And I am interpreting this as, b03, ndvi and the point vector data is important to the algorithm.

And then to determine exactly which areas (by coordinates) are important, that requires the unsupervised task on the raster data (whcih can be any raster data, not necessarily a converted .tif file)

[be-marc]

So let me see if I understand this correctly: the leipzig toy task has point data and other features as well. (I cannot find anywhere what b02, b03, etc are actually measuring but I am assuming something about each point.)

The features are the spectral bands (b02, b03, ...) of the Sentinel-2 image for each point. I also calculated the NDVI from them.

And I am interpreting this as, b03, ndvi and the point vector data is important to the algorithm.

No, the xy coordinates are not passed to the model.

tsk("leipzig")
<TaskClassifST:leipzig> (97 x 9)
* Target: land_cover
* Properties: multiclass
* Features (8):
  - dbl (8): b02, b03, b04, b06, b07, b08, b11, ndvi
* Coordinates:
           X       Y
 1: 732480.1 5693957
 2: 732217.4 5692769
 3: 732737.2 5692469
 4: 733169.3 5692777
 5: 732202.2 5692644
---                 
93: 733018.7 5692342
94: 732551.4 5692887
95: 732520.4 5692589
96: 732542.2 5692204
97: 732437.8 5692300

The model is only fitted with the 8 features. The coordinates are usually used for spatial resampling.

And then to determine exactly which areas (by coordinates) are important, that requires the unsupervised task on the raster data (whcih can be any raster data, not necessarily a converted .tif file)

No. Feature selection is not used to determine important areas. The unsupervised task is used to predict a map with the previously fitted model.

[walinchus]

Ah okay I see that's very helpful. Last question: Can you add to the book that code that made the sample points into a raster? (So that readers can reproduce it?)

For example, in your book my understanding is that the unsupervised task takes the inputs of the 8 features then uses that to predict a map with the previously fitted model.

Here, I take the data I use for the supervised task and use the terra::rast code you use to turn that into a raster, which it does.

But then when I go to use the as_task_unsupervised function, I get an error that there are no cell values.

(I did download a satellite pic of this area, and turned it into a tiff, and it didn't work, but anyway I'm not trying to predict every pixel necessarily, I'd be happy to just see the points that predict_spatial() is concerned with.)

[pat-s]

Based on your comments it might be that you're new to (spatial) modeling. In case you haven't done so, Elements of statistical learning is a good resource to start and understand the bigger modeling picture.

I'd like to point out some general points that are important as I don't understand the idea behind some of your questions. Most of them should also be addressed directly or indirectly in the mlr3book chapter(s).

• The response of your training data is usually data which you gathered yourself (i.e. either you collected them in the field or somebody else did). Yes, you can also convert imagery information into points and use this as the response but most often this information is used as features.
• You predict your model usually to remote sensing imagery to extrapolate to remote areas

For example, in your book my understanding is that the unsupervised task takes the inputs of the 8 features then uses that to predict a map with the previously fitted model.

It takes the trained model as input and then predicts on the information stored in the raster image.

Here, I take the data I use for the supervised task and use the terra::rast code you use to turn that into a raster, which it does.

I don't see why you would do that. You should not mix training and prediction/test data (there is a difference between prediction and test data!).

I'd be happy to just see the points that predict_spatial() is concerned with

What do you mean by "concerned with"? I still have the strong feeling you're having a wrong view on what you want to do or how the data is handled in modeling.

---

Some general points:

• Please don't post screenshots for code, ever and anywhere. Format your code in code blocks and create a reproducible example, preferably using the reprex package.
• Always post in public forums (e.g. GH oder stackoverflow) and not via email - no matter which project.
• There is more than training and prediction. You should tune your model and maybe use feature selection, though is really optional. You should estimate the performance of your model using an appropriate resampling strategy, most often in a nested-CV setting. If this sounds like a lot, I again refer to the book mentioned above to understand the bigger picture better. The biggest mistake you can make is to just execute a few functions of a package without knowing what is happening or if you did things right on the theory side.

[walinchus]

Thank you that is very helpful. I will check out that book. I read Hefin Rhy's "Machine Learning with R, the Tidyverse and mlr" which is what the bulk of this project (and previous ones) are based on but since the switch to mlr3 some things have not worked which is why I read the mlr3 book.

To give you some background: I get mapping data from a county FTP site: https://downloads.accuglobe.schneidergis.com/lawrenceoh/

Lawrence_Geo <- sf::st_read("Parcels.shp")
"st_crs"(Lawrence_Geo) <-"+proj=lcc +lat_1=38.73333333333333 +lat_2=40.03333333333333 +lat_0=38 +lon_0=-82.5 +x_0=600000.0000000001 +y_0=0 +datum=NAD83 +units=us-ft +no_defs" #addingCRS

Then I filter that with tax data to match the number of delinquents. And then I label them, based on the parcel number, as a land bank property or not. In other words, the machine learning problem is: out of all the delinquent properties, can we predict which will go into the land bank?

This final data set is here:
Land_Bank_Data.csv

This works great for the machine learning task:

data_sf = sf::st_as_sf(Lawrence_Centers, coords = c("x", "y"), crs = 4326)

task = as_task_classif_st(data_sf, id = "Lawrence_Spatial_Task", target = "Land_Bank", positive = "1" )

print(task)

learner = lrn("classif.randomForest", predict_type = "prob")
resampling_sp = rsmp("repeated_spcv_coords", folds = 4, repeats = 2)
rr_sp = resample(
  task = task, learner = learner,
  resampling = resampling_sp)

rr_sp$aggregate(measures = msr("classif.ce"))
rr_sp$aggregate(measures = msr("classif.acc"))

Then for spatial prediction, I put that into an unsupervised learner to create the tsk_predict but that doesn't work:

lawrence_raster = terra::rast(data_sf)
tsk_predict = as_task_unsupervised(lawrence_raster)

To clarify, I'm not trying to ask you to fix the code, I'm sure the code works fine if you can get the input into the right format. I'm looking for how to get it into that format that makes sense for the unsupervised task.

In other words, the book goes:

• create classification task using point data -> create raster -> use that raster to create an unsupervised learner that you can map

I create a raster with my data, using terra::rast as per the book's direction and that part gives me no errors. But then the unsupervised learner does not work, and that part requires a raster as per ?predict_spatial.

To put it another way: How would you visualize the landslide predictions in the Ecuador dataset in the the book?

[be-marc]

It seems that you don't have any raster data. You also want to predict on point data, right?

library(mlr3spatial)

data("leipzig", package = "mlr3spatial")

# train task based on point data
vector = sf::read_sf(system.file("extdata", "leipzig_points.gpkg", package = "mlr3spatial"), stringsAsFactors = TRUE)
task = as_task_classif_st(vector, id = "leipzig", target = "land_cover")

learner = lrn("classif.rpart")
learner$train(task)

# predict task based on point data
vector$land_cover = NULL # remove response
task_predict = as_task_unsupervised(vector)

pred = predict_spatial(task_predict, learner)

plot(pred)

You have to install the lastest dev version to run the example remotes::install_github("mlr-org/mlr3spatial").

To put it another way: How would you visualize the landslide predictions in the Ecuador dataset in the the book?

We have point data (for training) and raster data (for prediction) for the area of interest. So we visualize the landslide susceptibility in the whole area.

[pat-s]

Then for spatial prediction, I put that into an unsupervised learner to create the tsk_predict but that doesn't work

You're confusing things here. There is no unsupervised learner here, you wrap the raster image into an unsupervised task. The task holds your data. You predict to your data with your trained model (which is usually done in a supervised setting).

create classification task using point data -> create raster -> use that raster to create an unsupervised learner that you can map

See above, no unsupervised learner is needed in any process here.
Also you don't create the raster from your point data, these two datasets are usually completely distinct from each other.
The whole purpose of a prediction is to "create" values for a specific variable in a location (seen from a spatial perspective) for which you don't have data for that variable.

[walinchus]

@be-marc: Oh yes this is fantastic! I suggest adding this to the book.

@pat-s: You are right I meant to write "task" there not "learner." Sorry about that.