Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

pgraster fails to render data on node-mapnik in Lambda #969

Open
knutole opened this issue Apr 7, 2021 · 0 comments
Open

pgraster fails to render data on node-mapnik in Lambda #969

knutole opened this issue Apr 7, 2021 · 0 comments

Comments

@knutole
Copy link

knutole commented Apr 7, 2021

Using Mapnik on Lambda, with a PostGIS database which has already worked well with pgraster tile creation. However, since the migration to Lambda, the pgraster fails to render anything but empty tiles. (Regular raster tiles from PostGIS work well.)

There are no error messages, and all seems fine. (I can even query the dataset on any points, and correct data is returned from PostGIS.)

The only main difference from the previously working pgraster setup is really the Mapnik version, which now is 4.5.6. Has there been any special changes to Mapnik recently, which could effect pgraster?

Is there anything that looks wrong below, regarding rendering data with pgraster? (Note that the below code works well for postgis data.)


Logs, code, etc..

Pseudo-code for tile rendering in Lambda.. (removed error handling, saving, etc).

module.exports.raster = function (event, context) {

    // register mapnik modules
    mapnik.register_default_fonts();
    mapnik.register_default_input_plugins();
    mapnik.register_datasource(path.join(mapnik.settings.paths.input_plugins,'postgis.input'));
    mapnik.register_datasource(path.join(mapnik.settings.paths.input_plugins,'pgraster.input')); // <- no error
    console.log('Mapnik version:', mapnik.version); // 4.5.6

    // create layer
    var layer = new mapnik.Layer('layer', data.proj);

    // create datasource
    var datasource = new mapnik.Datasource(data.postgis_settings); // <- see below for content
   
    // set datasource
    layer.datasource = datasource;

    // add styles
    layer.styles = ['layer']; 

    // create map
    var map = new mapnik.Map(256, 256, data.proj);

    // set buffer
    map.bufferSize = data.bufferSize;

    // set extent
    map.extent = data.bbox; 

    // add layer to map
    map.add_layer(layer);

    // add xml (async)
    map.fromString(data.xml, {strict : true}, function (err, new_map) {        // <- see below for xml

        // map options
        var map_options = {
            buffer_size : data.bufferSize,
            variables : { 
                zoom : data.params.z 
            }
        }
        
        // create image
        var im = new mapnik.Image(256, 256);

        // render image
        new_map.render(im, map_options, function (err, tile) {
           
            tile.encode('png8', function (err, buffer) {
                // ok 
            });
        });
    });
};
// postgis settings passed to `mapnik.DataSource`
{
    user: 'REDACTED',
    password: 'REDACTED',
    host: 'REDACTED',
    srid: 3857,
    dbname: 'REDACTED',
    extent: [
        -0.692945604518282,
        57.4908604280581,
        32.9024354575735,
        71.8637803273957
    ],
    type: 'pgraster',
    table: 'file_jdajlukscwpmsvkcgegb',
    geometry_field: 'rast',
    asynchronous_request: false,
    clip_rasters: 'true',
    use_overviews: 'true',
    prescale_rasters: 'true',
    band: 1 // or 0, trying both
}
<?xml version="1.0" encoding="utf-8"?>
<!DOCTYPE Map[]>
<Map>
    <Style name="layer" filter-mode="first">
        <Rule>
            <RasterSymbolizer opacity="1" default-mode="linear" default-color="rgba(0, 0, 0, 0)" comp-op="color-dodge">
                <stop value="0" color="#ffffff" />
                <stop value="200" color="#ff00ff" />
                <stop value="201" color="rgba(0, 0, 0, 0)" />
                <stop value="255" color="rgba(0, 0, 0, 0)" mode="exact" />
            </RasterSymbolizer>
        </Rule>
    </Style>
    <Layer name="layer" srs="+proj=merc +a=6378137 +b=6378137 +lat_ts=0.0 +lon_0=0.0 +x_0=0.0 +y_0=0 +k=1.0 +units=m +nadgrids=@null +wktext +no_defs +over">
        <StyleName>layer</StyleName> 
    </Layer>
</Map>

PostGIS logs:

2021-04-07 13:26:07 UTC [28096-3] systemapic@mftrdjojda LOG:  statement: select r.r_table_schema sch, r.r_table_name tab, r.r_raster_column col, greatest(abs(r.scale_x), abs(r.scale_y)) scl from raster_overviews o, raster_columns r where o.r_table_schema = 'public' and o.r_table_name = 'file_jdajlukscwpmsvkcgegb' and o.r_raster_column = 'rast' and r.r_table_schema = o.o_table_schema and r.r_table_name = o.o_table_name and r.r_raster_column = o.o_raster_column ORDER BY scl ASC
2021-04-07 13:26:07 UTC [28097-3] systemapic@mftrdjojda LOG:  statement: select r.r_table_schema sch, r.r_table_name tab, r.r_raster_column col, greatest(abs(r.scale_x), abs(r.scale_y)) scl from raster_overviews o, raster_columns r where o.r_table_schema = 'public' and o.r_table_name = 'file_jdajlukscwpmsvkcgegb' and o.r_raster_column = 'rast' and r.r_table_schema = o.o_table_schema and r.r_table_name = o.o_table_name and r.r_raster_column = o.o_raster_column ORDER BY scl ASC
2021-04-07 13:26:07 UTC [28102-4] systemapic@mftrdjojda LOG:  statement: SELECT * FROM file_jdajlukscwpmsvkcgegb LIMIT 0
2021-04-07 13:26:07 UTC [28104-4] systemapic@mftrdjojda LOG:  statement: SELECT * FROM file_jdajlukscwpmsvkcgegb LIMIT 0
2021-04-07 13:26:07 UTC [28103-4] systemapic@mftrdjojda LOG:  statement: SELECT * FROM file_jdajlukscwpmsvkcgegb LIMIT 0
2021-04-07 13:26:07 UTC [28094-4] systemapic@mftrdjojda LOG:  statement: SELECT * FROM file_jdajlukscwpmsvkcgegb LIMIT 0
2021-04-07 13:26:07 UTC [28095-4] systemapic@mftrdjojda LOG:  statement: SELECT * FROM file_jdajlukscwpmsvkcgegb LIMIT 0
2021-04-07 13:26:07 UTC [28100-4] systemapic@mftrdjojda LOG:  statement: SELECT * FROM file_jdajlukscwpmsvkcgegb LIMIT 0
2021-04-07 13:26:07 UTC [28099-4] systemapic@mftrdjojda LOG:  statement: SELECT * FROM file_jdajlukscwpmsvkcgegb LIMIT 0
2021-04-07 13:26:07 UTC [28098-4] systemapic@mftrdjojda LOG:  statement: SELECT * FROM file_jdajlukscwpmsvkcgegb LIMIT 0
2021-04-07 13:26:07 UTC [28096-4] systemapic@mftrdjojda LOG:  statement: SELECT * FROM file_jdajlukscwpmsvkcgegb LIMIT 0
2021-04-07 13:26:07 UTC [28097-4] systemapic@mftrdjojda LOG:  statement: SELECT * FROM file_jdajlukscwpmsvkcgegb LIMIT 0
2021-04-07 13:26:07 UTC [28101-4] systemapic@mftrdjojda LOG:  statement: SELECT * FROM file_jdajlukscwpmsvkcgegb LIMIT 0
2021-04-07 13:26:08 UTC [28105-1] systemapic@mftrdjojda LOG:  statement: SET DEFAULT_TRANSACTION_READ_ONLY = TRUE; SET CLIENT_MIN_MESSAGES = WARNING;
2021-04-07 13:26:08 UTC [28105-2] systemapic@mftrdjojda LOG:  statement: SELECT r_raster_column col, srid, r_table_schema FROM raster_columns WHERE r_table_name='file_jdajlukscwpmsvkcgegb'
2021-04-07 13:26:08 UTC [28105-3] systemapic@mftrdjojda LOG:  statement: select r.r_table_schema sch, r.r_table_name tab, r.r_raster_column col, greatest(abs(r.scale_x), abs(r.scale_y)) scl from raster_overviews o, raster_columns r where o.r_table_schema = 'public' and o.r_table_name = 'file_jdajlukscwpmsvkcgegb' and o.r_raster_column = 'rast' and r.r_table_schema = o.o_table_schema and r.r_table_name = o.o_table_name and r.r_raster_column = o.o_raster_column ORDER BY scl ASC
2021-04-07 13:26:08 UTC [28105-4] systemapic@mftrdjojda LOG:  statement: SELECT * FROM file_jdajlukscwpmsvkcgegb LIMIT 0
2021-04-07 13:26:08 UTC [28106-1] systemapic@mftrdjojda LOG:  statement: SET DEFAULT_TRANSACTION_READ_ONLY = TRUE; SET CLIENT_MIN_MESSAGES = WARNING;
2021-04-07 13:26:08 UTC [28106-2] systemapic@mftrdjojda LOG:  statement: SELECT r_raster_column col, srid, r_table_schema FROM raster_columns WHERE r_table_name='file_jdajlukscwpmsvkcgegb'
2021-04-07 13:26:08 UTC [28106-3] systemapic@mftrdjojda LOG:  statement: select r.r_table_schema sch, r.r_table_name tab, r.r_raster_column col, greatest(abs(r.scale_x), abs(r.scale_y)) scl from raster_overviews o, raster_columns r where o.r_table_schema = 'public' and o.r_table_name = 'file_jdajlukscwpmsvkcgegb' and o.r_raster_column = 'rast' and r.r_table_schema = o.o_table_schema and r.r_table_name = o.o_table_name and r.r_raster_column = o.o_raster_column ORDER BY scl ASC
2021-04-07 13:26:08 UTC [28106-4] systemapic@mftrdjojda LOG:  statement: SELECT * FROM file_jdajlukscwpmsvkcgegb LIMIT 0
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant