Can Gdal.Translate() return an object instead of writing a file?
1
I have some Python GDAL code I am using to convert a GeoTIFF file to PNG. The code uses the gdal.Translate() function which generates a new file each time I run it. However, I was wondering if there is a way to get the Python gdal.Translate() function to return a python object instead of writing a file? Ideally I would like to write to a numpy array or something, but any kind of internal object would be fine.
Here is some sample code I am using:
from osgeo import gdal
scale = '-scale min_val max_val'
options_list = [
'-ot Byte',
'-of JPEG',
scale
]
options_string = " ".join(options_list)
gdal.Translate('test.jpg',
'test.tif',
options=options_string)
I looked over the documentation and source code to figure out how gdal.Translate() worked, but could not get past the TranslateInternal() function. I could not find a link through the source code past that.
Any suggestions would be appreciated.
python gdal gdal-translate
asked 6 hours ago
krishnabkrishnab
248214
add a comment |
1
I have some Python GDAL code I am using to convert a GeoTIFF file to PNG. The code uses the gdal.Translate() function which generates a new file each time I run it. However, I was wondering if there is a way to get the Python gdal.Translate() function to return a python object instead of writing a file? Ideally I would like to write to a numpy array or something, but any kind of internal object would be fine.
Here is some sample code I am using:
from osgeo import gdal
scale = '-scale min_val max_val'
options_list = [
'-ot Byte',
'-of JPEG',
scale
]
options_string = " ".join(options_list)
gdal.Translate('test.jpg',
'test.tif',
options=options_string)
I looked over the documentation and source code to figure out how gdal.Translate() worked, but could not get past the TranslateInternal() function. I could not find a link through the source code past that.
Any suggestions would be appreciated.
python gdal gdal-translate
asked 6 hours ago
krishnabkrishnab
248214
There is a memory driver which may help gdal.org/frmt_mem.html I'm not sure how you could leverage the pointer in python but it works great in C++ (until you create a raster larger than your available memory). TranslateInternal() would be calling a C external link, you could find it in the source code if you understand ANSI C/C++. It might be best to write to a file in your os.environ.get('temp') then gdal.Open the temp raster.
– Michael Stimson
6 hours ago
add a comment |
1
1
1
I have some Python GDAL code I am using to convert a GeoTIFF file to PNG. The code uses the gdal.Translate() function which generates a new file each time I run it. However, I was wondering if there is a way to get the Python gdal.Translate() function to return a python object instead of writing a file? Ideally I would like to write to a numpy array or something, but any kind of internal object would be fine.
Here is some sample code I am using:
from osgeo import gdal
scale = '-scale min_val max_val'
options_list = [
'-ot Byte',
'-of JPEG',
scale
]
options_string = " ".join(options_list)
gdal.Translate('test.jpg',
'test.tif',
options=options_string)
I looked over the documentation and source code to figure out how gdal.Translate() worked, but could not get past the TranslateInternal() function. I could not find a link through the source code past that.
Any suggestions would be appreciated.
python gdal gdal-translate
asked 6 hours ago
krishnabkrishnab
248214
I have some Python GDAL code I am using to convert a GeoTIFF file to PNG. The code uses the gdal.Translate() function which generates a new file each time I run it. However, I was wondering if there is a way to get the Python gdal.Translate() function to return a python object instead of writing a file? Ideally I would like to write to a numpy array or something, but any kind of internal object would be fine.
Here is some sample code I am using:
from osgeo import gdal
scale = '-scale min_val max_val'
options_list = [
'-ot Byte',
'-of JPEG',
scale
]
options_string = " ".join(options_list)
gdal.Translate('test.jpg',
'test.tif',
options=options_string)
I looked over the documentation and source code to figure out how gdal.Translate() worked, but could not get past the TranslateInternal() function. I could not find a link through the source code past that.
Any suggestions would be appreciated.
python gdal gdal-translate
python gdal gdal-translate
asked 6 hours ago
krishnabkrishnab
248214
asked 6 hours ago
krishnabkrishnab
248214
asked 6 hours ago
krishnabkrishnab
248214
asked 6 hours ago
krishnabkrishnab
248214
asked 6 hours ago
krishnabkrishnab
248214
248214
There is a memory driver which may help gdal.org/frmt_mem.html I'm not sure how you could leverage the pointer in python but it works great in C++ (until you create a raster larger than your available memory). TranslateInternal() would be calling a C external link, you could find it in the source code if you understand ANSI C/C++. It might be best to write to a file in your os.environ.get('temp') then gdal.Open the temp raster.
– Michael Stimson
6 hours ago
add a comment |
There is a memory driver which may help gdal.org/frmt_mem.html I'm not sure how you could leverage the pointer in python but it works great in C++ (until you create a raster larger than your available memory). TranslateInternal() would be calling a C external link, you could find it in the source code if you understand ANSI C/C++. It might be best to write to a file in your os.environ.get('temp') then gdal.Open the temp raster.
– Michael Stimson
6 hours ago
There is a memory driver which may help gdal.org/frmt_mem.html I'm not sure how you could leverage the pointer in python but it works great in C++ (until you create a raster larger than your available memory). TranslateInternal() would be calling a C external link, you could find it in the source code if you understand ANSI C/C++. It might be best to write to a file in your os.environ.get('temp') then gdal.Open the temp raster.
– Michael Stimson
6 hours ago
There is a memory driver which may help gdal.org/frmt_mem.html I'm not sure how you could leverage the pointer in python but it works great in C++ (until you create a raster larger than your available memory). TranslateInternal() would be calling a C external link, you could find it in the source code if you understand ANSI C/C++. It might be best to write to a file in your os.environ.get('temp') then gdal.Open the temp raster.
– Michael Stimson
6 hours ago
add a comment |
1 Answer
1
active
oldest
votes
3
gdal.Translate does return an object, it returns a gdal.Dataset object.
from osgeo import gdal
in_ds = gdal.OpenEx('path/to/input.tif')
print(in_ds)
<osgeo.gdal.Dataset; proxy of <Swig Object of type 'GDALDatasetShadow *' at 0x00000000037BE930> >
out_ds = gdal.Translate('path/to/output.tif', in_ds)
print(out_ds )
<osgeo.gdal.Dataset; proxy of <Swig Object of type 'GDALDatasetShadow *' at 0x00000000037BE990> >
It has to write the translated output somewhere...
If you don't want to write to disk, write to memory (don't bother with the MEM driver), use the "VSIMEM" virtual filesystem to write a GeoTIFF to memory:
out_ds = gdal.Translate('/vsimem/in_memory_output.tif', in_ds)
You can then read it into a numpy array if you want.
out_arr = out_ds.ReadAsArray()
answered 6 hours ago
user2856user2856
30.4k258106
Oh this is great. Yeah, I will give this a try. This will make stuff a lot easier. Just to anticipate my next question, can I convert aDatasetobject to a numpy array using theds.GetRasterBand(1).ReadAsArray()function? Is that what you were thinking or am I thinking in the wrong direction.
– krishnab
5 hours ago
Oh thank you so much. I was trying to figure this out since yesterday. Haha, like if I had energy left I would thank you more :). Yeah, so that will load all the bands into an array. I was trying to figure out if I needed to manually stack each band into an RGB array or shift from a raster ordering of the data to an image ordering, etc. But this totally answers my question.
– krishnab
5 hours ago
Yep, that is what I figured. I can use therasterio.plot.reshape_as_image()function to then reorder it into row, column, bands order :).
– krishnab
5 hours ago
Ahh, good tip. No I am not writing out to a raster again. I am taking the TIFF data and encoding it to aTFRecordsformat for some machine learning in Tensorflow. Kinda a long story but trying to be systematic about it. So I need to take the TIFF data, convert it to 8bit format, encode it as a string and write it to TFRecords. But then I need to decode the data from string in Tensorflow and validate the data went in properly. So I need to be able to check the decoded data and that is why I was using a PNG or JPEG format since they are easy to plot.
– krishnab
5 hours ago
Haha, I know it is not the easiest and that conversion from 16bit TIFF to 8bit loses information. But just need to get the basic prototype working and then can make the necessary fixes to do it right.
– krishnab
5 hours ago
add a comment |
Your Answer
StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "79"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});
function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: false,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: null,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});
}
});
draft saved
draft discarded
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fgis.stackexchange.com%2fquestions%2f316034%2fcan-gdal-translate-return-an-object-instead-of-writing-a-file%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
1 Answer
1
active
oldest
votes
1 Answer
1
active
oldest
votes
active
oldest
votes
active
oldest
votes
3
gdal.Translate does return an object, it returns a gdal.Dataset object.
from osgeo import gdal
in_ds = gdal.OpenEx('path/to/input.tif')
print(in_ds)
<osgeo.gdal.Dataset; proxy of <Swig Object of type 'GDALDatasetShadow *' at 0x00000000037BE930> >
out_ds = gdal.Translate('path/to/output.tif', in_ds)
print(out_ds )
<osgeo.gdal.Dataset; proxy of <Swig Object of type 'GDALDatasetShadow *' at 0x00000000037BE990> >
It has to write the translated output somewhere...
If you don't want to write to disk, write to memory (don't bother with the MEM driver), use the "VSIMEM" virtual filesystem to write a GeoTIFF to memory:
out_ds = gdal.Translate('/vsimem/in_memory_output.tif', in_ds)
You can then read it into a numpy array if you want.
out_arr = out_ds.ReadAsArray()
answered 6 hours ago
user2856user2856
30.4k258106
Oh this is great. Yeah, I will give this a try. This will make stuff a lot easier. Just to anticipate my next question, can I convert aDatasetobject to a numpy array using theds.GetRasterBand(1).ReadAsArray()function? Is that what you were thinking or am I thinking in the wrong direction.
– krishnab
5 hours ago
Oh thank you so much. I was trying to figure this out since yesterday. Haha, like if I had energy left I would thank you more :). Yeah, so that will load all the bands into an array. I was trying to figure out if I needed to manually stack each band into an RGB array or shift from a raster ordering of the data to an image ordering, etc. But this totally answers my question.
– krishnab
5 hours ago
Yep, that is what I figured. I can use therasterio.plot.reshape_as_image()function to then reorder it into row, column, bands order :).
– krishnab
5 hours ago
Ahh, good tip. No I am not writing out to a raster again. I am taking the TIFF data and encoding it to aTFRecordsformat for some machine learning in Tensorflow. Kinda a long story but trying to be systematic about it. So I need to take the TIFF data, convert it to 8bit format, encode it as a string and write it to TFRecords. But then I need to decode the data from string in Tensorflow and validate the data went in properly. So I need to be able to check the decoded data and that is why I was using a PNG or JPEG format since they are easy to plot.
– krishnab
5 hours ago
Haha, I know it is not the easiest and that conversion from 16bit TIFF to 8bit loses information. But just need to get the basic prototype working and then can make the necessary fixes to do it right.
– krishnab
5 hours ago
add a comment |
3
gdal.Translate does return an object, it returns a gdal.Dataset object.
from osgeo import gdal
in_ds = gdal.OpenEx('path/to/input.tif')
print(in_ds)
<osgeo.gdal.Dataset; proxy of <Swig Object of type 'GDALDatasetShadow *' at 0x00000000037BE930> >
out_ds = gdal.Translate('path/to/output.tif', in_ds)
print(out_ds )
<osgeo.gdal.Dataset; proxy of <Swig Object of type 'GDALDatasetShadow *' at 0x00000000037BE990> >
It has to write the translated output somewhere...
If you don't want to write to disk, write to memory (don't bother with the MEM driver), use the "VSIMEM" virtual filesystem to write a GeoTIFF to memory:
out_ds = gdal.Translate('/vsimem/in_memory_output.tif', in_ds)
You can then read it into a numpy array if you want.
out_arr = out_ds.ReadAsArray()
answered 6 hours ago
user2856user2856
30.4k258106
Oh this is great. Yeah, I will give this a try. This will make stuff a lot easier. Just to anticipate my next question, can I convert aDatasetobject to a numpy array using theds.GetRasterBand(1).ReadAsArray()function? Is that what you were thinking or am I thinking in the wrong direction.
– krishnab
5 hours ago
Oh thank you so much. I was trying to figure this out since yesterday. Haha, like if I had energy left I would thank you more :). Yeah, so that will load all the bands into an array. I was trying to figure out if I needed to manually stack each band into an RGB array or shift from a raster ordering of the data to an image ordering, etc. But this totally answers my question.
– krishnab
5 hours ago
Yep, that is what I figured. I can use therasterio.plot.reshape_as_image()function to then reorder it into row, column, bands order :).
– krishnab
5 hours ago
Ahh, good tip. No I am not writing out to a raster again. I am taking the TIFF data and encoding it to aTFRecordsformat for some machine learning in Tensorflow. Kinda a long story but trying to be systematic about it. So I need to take the TIFF data, convert it to 8bit format, encode it as a string and write it to TFRecords. But then I need to decode the data from string in Tensorflow and validate the data went in properly. So I need to be able to check the decoded data and that is why I was using a PNG or JPEG format since they are easy to plot.
– krishnab
5 hours ago
Haha, I know it is not the easiest and that conversion from 16bit TIFF to 8bit loses information. But just need to get the basic prototype working and then can make the necessary fixes to do it right.
– krishnab
5 hours ago
add a comment |
3
3
3
gdal.Translate does return an object, it returns a gdal.Dataset object.
from osgeo import gdal
in_ds = gdal.OpenEx('path/to/input.tif')
print(in_ds)
<osgeo.gdal.Dataset; proxy of <Swig Object of type 'GDALDatasetShadow *' at 0x00000000037BE930> >
out_ds = gdal.Translate('path/to/output.tif', in_ds)
print(out_ds )
<osgeo.gdal.Dataset; proxy of <Swig Object of type 'GDALDatasetShadow *' at 0x00000000037BE990> >
It has to write the translated output somewhere...
If you don't want to write to disk, write to memory (don't bother with the MEM driver), use the "VSIMEM" virtual filesystem to write a GeoTIFF to memory:
out_ds = gdal.Translate('/vsimem/in_memory_output.tif', in_ds)
You can then read it into a numpy array if you want.
out_arr = out_ds.ReadAsArray()
answered 6 hours ago
user2856user2856
30.4k258106
gdal.Translate does return an object, it returns a gdal.Dataset object.
from osgeo import gdal
in_ds = gdal.OpenEx('path/to/input.tif')
print(in_ds)
<osgeo.gdal.Dataset; proxy of <Swig Object of type 'GDALDatasetShadow *' at 0x00000000037BE930> >
out_ds = gdal.Translate('path/to/output.tif', in_ds)
print(out_ds )
<osgeo.gdal.Dataset; proxy of <Swig Object of type 'GDALDatasetShadow *' at 0x00000000037BE990> >
It has to write the translated output somewhere...
If you don't want to write to disk, write to memory (don't bother with the MEM driver), use the "VSIMEM" virtual filesystem to write a GeoTIFF to memory:
out_ds = gdal.Translate('/vsimem/in_memory_output.tif', in_ds)
You can then read it into a numpy array if you want.
out_arr = out_ds.ReadAsArray()
answered 6 hours ago
user2856user2856
30.4k258106
edited 5 hours ago
answered 6 hours ago
user2856user2856
30.4k258106
answered 6 hours ago
user2856user2856
30.4k258106
answered 6 hours ago
user2856user2856
30.4k258106
30.4k258106
Oh this is great. Yeah, I will give this a try. This will make stuff a lot easier. Just to anticipate my next question, can I convert aDatasetobject to a numpy array using theds.GetRasterBand(1).ReadAsArray()function? Is that what you were thinking or am I thinking in the wrong direction.
– krishnab
5 hours ago
Oh thank you so much. I was trying to figure this out since yesterday. Haha, like if I had energy left I would thank you more :). Yeah, so that will load all the bands into an array. I was trying to figure out if I needed to manually stack each band into an RGB array or shift from a raster ordering of the data to an image ordering, etc. But this totally answers my question.
– krishnab
5 hours ago
Yep, that is what I figured. I can use therasterio.plot.reshape_as_image()function to then reorder it into row, column, bands order :).
– krishnab
5 hours ago
Ahh, good tip. No I am not writing out to a raster again. I am taking the TIFF data and encoding it to aTFRecordsformat for some machine learning in Tensorflow. Kinda a long story but trying to be systematic about it. So I need to take the TIFF data, convert it to 8bit format, encode it as a string and write it to TFRecords. But then I need to decode the data from string in Tensorflow and validate the data went in properly. So I need to be able to check the decoded data and that is why I was using a PNG or JPEG format since they are easy to plot.
– krishnab
5 hours ago
Haha, I know it is not the easiest and that conversion from 16bit TIFF to 8bit loses information. But just need to get the basic prototype working and then can make the necessary fixes to do it right.
– krishnab
5 hours ago
add a comment |
Oh this is great. Yeah, I will give this a try. This will make stuff a lot easier. Just to anticipate my next question, can I convert aDatasetobject to a numpy array using theds.GetRasterBand(1).ReadAsArray()function? Is that what you were thinking or am I thinking in the wrong direction.
– krishnab
5 hours ago
Oh thank you so much. I was trying to figure this out since yesterday. Haha, like if I had energy left I would thank you more :). Yeah, so that will load all the bands into an array. I was trying to figure out if I needed to manually stack each band into an RGB array or shift from a raster ordering of the data to an image ordering, etc. But this totally answers my question.
– krishnab
5 hours ago
Yep, that is what I figured. I can use therasterio.plot.reshape_as_image()function to then reorder it into row, column, bands order :).
– krishnab
5 hours ago
Ahh, good tip. No I am not writing out to a raster again. I am taking the TIFF data and encoding it to aTFRecordsformat for some machine learning in Tensorflow. Kinda a long story but trying to be systematic about it. So I need to take the TIFF data, convert it to 8bit format, encode it as a string and write it to TFRecords. But then I need to decode the data from string in Tensorflow and validate the data went in properly. So I need to be able to check the decoded data and that is why I was using a PNG or JPEG format since they are easy to plot.
– krishnab
5 hours ago
Haha, I know it is not the easiest and that conversion from 16bit TIFF to 8bit loses information. But just need to get the basic prototype working and then can make the necessary fixes to do it right.
– krishnab
5 hours ago
Oh this is great. Yeah, I will give this a try. This will make stuff a lot easier. Just to anticipate my next question, can I convert a
Dataset object to a numpy array using the ds.GetRasterBand(1).ReadAsArray() function? Is that what you were thinking or am I thinking in the wrong direction.– krishnab
5 hours ago
Oh this is great. Yeah, I will give this a try. This will make stuff a lot easier. Just to anticipate my next question, can I convert a
Dataset object to a numpy array using the ds.GetRasterBand(1).ReadAsArray() function? Is that what you were thinking or am I thinking in the wrong direction.– krishnab
5 hours ago
Oh thank you so much. I was trying to figure this out since yesterday. Haha, like if I had energy left I would thank you more :). Yeah, so that will load all the bands into an array. I was trying to figure out if I needed to manually stack each band into an RGB array or shift from a raster ordering of the data to an image ordering, etc. But this totally answers my question.
– krishnab
5 hours ago
Oh thank you so much. I was trying to figure this out since yesterday. Haha, like if I had energy left I would thank you more :). Yeah, so that will load all the bands into an array. I was trying to figure out if I needed to manually stack each band into an RGB array or shift from a raster ordering of the data to an image ordering, etc. But this totally answers my question.
– krishnab
5 hours ago
Yep, that is what I figured. I can use the
rasterio.plot.reshape_as_image() function to then reorder it into row, column, bands order :).– krishnab
5 hours ago
Yep, that is what I figured. I can use the
rasterio.plot.reshape_as_image() function to then reorder it into row, column, bands order :).– krishnab
5 hours ago
Ahh, good tip. No I am not writing out to a raster again. I am taking the TIFF data and encoding it to a
TFRecords format for some machine learning in Tensorflow. Kinda a long story but trying to be systematic about it. So I need to take the TIFF data, convert it to 8bit format, encode it as a string and write it to TFRecords. But then I need to decode the data from string in Tensorflow and validate the data went in properly. So I need to be able to check the decoded data and that is why I was using a PNG or JPEG format since they are easy to plot.– krishnab
5 hours ago
Ahh, good tip. No I am not writing out to a raster again. I am taking the TIFF data and encoding it to a
TFRecords format for some machine learning in Tensorflow. Kinda a long story but trying to be systematic about it. So I need to take the TIFF data, convert it to 8bit format, encode it as a string and write it to TFRecords. But then I need to decode the data from string in Tensorflow and validate the data went in properly. So I need to be able to check the decoded data and that is why I was using a PNG or JPEG format since they are easy to plot.– krishnab
5 hours ago
Haha, I know it is not the easiest and that conversion from 16bit TIFF to 8bit loses information. But just need to get the basic prototype working and then can make the necessary fixes to do it right.
– krishnab
5 hours ago
Haha, I know it is not the easiest and that conversion from 16bit TIFF to 8bit loses information. But just need to get the basic prototype working and then can make the necessary fixes to do it right.
– krishnab
5 hours ago
add a comment |
draft saved
draft discarded
Thanks for contributing an answer to Geographic Information Systems Stack Exchange!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
draft saved
draft discarded
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fgis.stackexchange.com%2fquestions%2f316034%2fcan-gdal-translate-return-an-object-instead-of-writing-a-file%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
There is a memory driver which may help gdal.org/frmt_mem.html I'm not sure how you could leverage the pointer in python but it works great in C++ (until you create a raster larger than your available memory). TranslateInternal() would be calling a C external link, you could find it in the source code if you understand ANSI C/C++. It might be best to write to a file in your os.environ.get('temp') then gdal.Open the temp raster.
– Michael Stimson
6 hours ago