mappool.c 16.9 KB
Newer Older
1
/******************************************************************************
2
 * $Id$
3 4 5 6 7 8 9 10 11 12 13 14 15 16 17
 *
 * Project:  MapServer
 * Purpose:  Implement new style connection pooling.
 * Author:   Frank Warmerdam, warmerdam@pobox.com
 *
 ******************************************************************************
 * Copyright (c) 1996-2005 Regents of the University of Minnesota.
 *
 * Permission is hereby granted, free of charge, to any person obtaining a
 * copy of this software and associated documentation files (the "Software"),
 * to deal in the Software without restriction, including without limitation
 * the rights to use, copy, modify, merge, publish, distribute, sublicense,
 * and/or sell copies of the Software, and to permit persons to whom the
 * Software is furnished to do so, subject to the following conditions:
 *
18
 * The above copyright notice and this permission notice shall be included in
19 20 21 22 23 24 25 26 27 28 29
 * all copies of this Software or works derived from this Software.
 *
 * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS
 * OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
 * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
 * THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
 * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
 * FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER
 * DEALINGS IN THE SOFTWARE.
 ******************************************************************************

30
             New MapServer Connection Pooling
31 32 33 34
             ================================

This attempts to describe how the new connection pooling support (introduced
Sept/2004) works and what the maintainer of a connection type needs to do
35
to take advantage of it.
36 37 38 39 40

First, the new connection pooling support makes the assumption that
a connection can be abstracted as a void pointer.  Further it assumes that
the connection identify is unique defined by the connection string and
the connection type on layers.  So if two layers have the same connection
41
type and connection string that they can share a connection handle.
42 43 44 45 46

The old connection sharing was based on searching previous layers in the
same map for a layer to copy an existing connection from.  That was ok for
sharing connection between layers in the same map, but didn't address the
need to share connections over a longer term as is the case in a FastCGI
47
situation where the cgi is long running and handles many CGI requests.
48 49 50 51 52 53 54

The mapool.c code essentially maintains a cache of open connections with the
following information for each:
  - connectiontype / connection
  - connection handle
  - reference count
  - life span indicator
55
  - callback function to close the connection.
56 57 58

The life span indicator is controlled by the CLOSE_CONNECTION PROCESSING
option on the layer(s).  If this is set to NORMAL (or not set at all) the
59
connection will be closed as soon as the reference count drops to zero.  This
60 61 62 63 64 65
is MS_LIFE_ZEROREF, and in normal use results in connections only be shared
between layers in a given map as the connection will be closed when no layers
are open against it anymore.  The other possible value for the CLOSE_CONNECTION
option is DEFER which basically results in the connection being kept open
till the application closes (msCleanup() will ensure the connection is closed
on exit if called).  This case is called MS_LIFE_FOREVER.
66 67
The CLOSE_CONNECTION=ALWAYS setting provides to suppress the connection pooling
for a particular layer. In this case the MS_LIFE_SINGLE setting is used, which
68
ensures that a new connection is created for each request and it is always
69
closed when the connection is released. This kind of connection cannot be reused.
70 71 72

The callback is a function provided with the connection handle when it is
registered.  It takes a single "void *" argument which is the connection
73
handle.
74 75 76 77 78

Updating a Driver
-----------------

The following are the steps to ensure a particular format/database supports
79
connection pooling (ie. mappostgis.c, maporacle.c, mapogr.cpp, etc)
80
We will use POSTGIS names for convenience.
81 82

1) in msPOSTGISLayerOpen() call msConnPoolRequest(layer) in order to get
83
   a database connection.
84 85 86

        layerinfo->conn = (PGconn *) msConnPoolRequest( layer );

87
2) In msPOSTGISLayerOpen(): if msConnPoolRequest() returned NULL then
88 89 90
   manually open a connection to the database (ie. PQconnectcb()) and then
   register this handle with the pool API by calling msmsConnPoolRegister().

91
      layerinfo->conn = PQconnectdb( layer->connection );
92 93 94 95

            if (PQstatus(layerinfo->conn) == CONNECTION_BAD)
               <report error>
            else
96
                msConnPoolRegister( layer, layerinfo->conn,
97 98
                                    msPOSTGISCloseConnection );

99 100
3) Implement a callback function (msPOSTGISCloseConnection) that the
   connection pooling API can call when it wants to close the connection.
101 102 103 104 105 106 107

   static void msPOSTGISCloseConnection( void *conn_handle )

   {
       PQfinish( (PGconn*) conn_handle );
   }

108 109
4) In msPOSTGISLayerClose() release the connection handle instead of
   directly closing it.
110 111 112 113

            msConnPoolRelease( layer, layerinfo->conn );
            layerinfo->conn = NULL;

114
5) If there was any use of msCheckConnection() or the "sameconnection"
115 116 117 118 119 120 121 122
   member of the layerObj, then old style connection pooling is already
   present.  Remove it.

Thats it!

Other Notes
-----------

123
o The connection pooling API will report details about connection
124
  registrations, requests, releases and closes if the layer debug flag is
125
  on for the layers in question.
126 127 128 129

o The connection pooling API will let a connection be used/referenced multiple
  times from a single thread, but will not allow a connection to be shared
  between different threads concurrently.  But if a connection is released
130
  by one thread, it is available for use by another thread.
131 132 133 134 135 136 137

 ****************************************************************************/

#include "mapserver.h"
#include "mapthread.h"


138 139

/* defines for lifetime.
140 141 142 143
   A positive number is a time-from-last use in seconds */

#define MS_LIFE_FOREVER       -1
#define MS_LIFE_ZEROREF       -2
144
#define MS_LIFE_SINGLE        -3
145 146

typedef struct {
147 148 149 150 151
  enum MS_CONNECTION_TYPE connectiontype;
  char *connection;

  int   lifespan;
  int   ref_count;
152
  void*   thread_id;
153
  int   debug;
154

155
  time_t last_used;
156

157
  void  *conn_handle;
158

159
  void  (*close)( void * );
160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175
} connectionObj;

/*
** These static structures are protected by the TLOCK_POOL mutex.
*/

static int connectionCount = 0;
static int connectionMax = 0;
static connectionObj *connections = NULL;

/************************************************************************/
/*                         msConnPoolRegister()                         */
/*                                                                      */
/*      Register a new connection with the connection pool tracker.     */
/************************************************************************/

176 177
void msConnPoolRegister( layerObj *layer,
                         void *conn_handle,
178 179 180
                         void (*close_func)( void * ) )

{
181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204
  const char *close_connection = NULL;
  connectionObj *conn = NULL;

  if( layer->debug )
    msDebug( "msConnPoolRegister(%s,%s,%p)\n",
             layer->name, layer->connection, conn_handle );

  /* -------------------------------------------------------------------- */
  /*      We can't meaningful keep a connection with no connection or     */
  /*      connection type string on the layer.                            */
  /* -------------------------------------------------------------------- */
  if( layer->connection == NULL ) {
    if( layer->tileindex != NULL
        && layer->connectiontype == MS_OGR ) {
      /* this is ok, no need to make a fuss */
    } else {
      msDebug( "%s: Missing CONNECTION on layer %s.\n",
               "msConnPoolRegister()",
               layer->name );

      msSetError( MS_MISCERR,
                  "Missing CONNECTION on layer %s.",
                  "msConnPoolRegister()",
                  layer->name );
205
    }
206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222
    return;
  }

  /* -------------------------------------------------------------------- */
  /*      Grow the array of connection information objects if needed.     */
  /* -------------------------------------------------------------------- */
  msAcquireLock( TLOCK_POOL );

  if( connectionCount == connectionMax ) {
    connectionMax += 10;
    connections = (connectionObj *)
                  realloc(connections,
                          sizeof(connectionObj) * connectionMax );
    if( connections == NULL ) {
      msSetError(MS_MEMERR, NULL, "msConnPoolRegister()");
      msReleaseLock( TLOCK_POOL );
      return;
223
    }
224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263
  }

  /* -------------------------------------------------------------------- */
  /*      Set the new connection information.                             */
  /* -------------------------------------------------------------------- */
  conn = connections + connectionCount;

  connectionCount++;

  conn->connectiontype = layer->connectiontype;
  conn->connection = msStrdup( layer->connection );
  conn->close = close_func;
  conn->ref_count = 1;
  conn->thread_id = msGetThreadId();
  conn->last_used = time(NULL);
  conn->conn_handle = conn_handle;
  conn->debug = layer->debug;

  /* -------------------------------------------------------------------- */
  /*      Categorize the connection handling information.                 */
  /* -------------------------------------------------------------------- */
  close_connection =
    msLayerGetProcessingKey( layer, "CLOSE_CONNECTION" );

  if( close_connection == NULL )
    close_connection = "NORMAL";

  if( strcasecmp(close_connection,"NORMAL") == 0 )
    conn->lifespan = MS_LIFE_ZEROREF;
  else if( strcasecmp(close_connection,"DEFER") == 0 )
    conn->lifespan = MS_LIFE_FOREVER;
  else if( strcasecmp(close_connection,"ALWAYS") == 0 )
    conn->lifespan = MS_LIFE_SINGLE;
  else {
    msDebug("msConnPoolRegister(): "
            "Unrecognised CLOSE_CONNECTION value '%s'\n",
            close_connection );

    msSetError( MS_MISCERR, "Unrecognised CLOSE_CONNECTION value '%s'",
                "msConnPoolRegister()",
264
                close_connection );
265 266
    conn->lifespan = MS_LIFE_ZEROREF;
  }
267

268
  msReleaseLock( TLOCK_POOL );
269 270 271 272 273 274 275 276 277 278 279 280 281
}

/************************************************************************/
/*                          msConnPoolClose()                           */
/*                                                                      */
/*      Close the indicated connection.  The index in the connection    */
/*      table is passed.  Remove the connection from the table as       */
/*      well.                                                           */
/************************************************************************/

static void msConnPoolClose( int conn_index )

{
282
  connectionObj *conn = connections + conn_index;
283

284
  if( conn->ref_count > 0 ) {
285
    if( conn->debug )
286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318
      msDebug( "msConnPoolClose(): "
               "Closing connection %s even though ref_count=%d.\n",
               conn->connection, conn->ref_count );

    msSetError( MS_MISCERR,
                "Closing connection %s even though ref_count=%d.",
                "msConnPoolClose()",
                conn->connection,
                conn->ref_count );
  }

  if( conn->debug )
    msDebug( "msConnPoolClose(%s,%p)\n",
             conn->connection, conn->conn_handle );

  if( conn->close != NULL )
    conn->close( conn->conn_handle );

  /* free malloced() stuff in this connection */
  free( conn->connection );

  connectionCount--;
  if( connectionCount == 0 ) {
    /* if there are no connections left we will "cleanup".  */
    connectionMax = 0;
    free( connections );
    connections = NULL;
  } else {
    /* move the last connection in place of our now closed one */
    memcpy( connections + conn_index,
            connections + connectionCount,
            sizeof(connectionObj) );
  }
319 320 321 322 323 324 325 326 327 328 329 330 331 332
}

/************************************************************************/
/*                         msConnPoolRequest()                          */
/*                                                                      */
/*      Ask for a connection from the connection pool for use with      */
/*      the current layer.  If found (CONNECTION and CONNECTIONTYPE     */
/*      match) then return it and up the ref count.  Otherwise          */
/*      return NULL.                                                    */
/************************************************************************/

void *msConnPoolRequest( layerObj *layer )

{
333 334
  int  i;
  const char* close_connection;
335

336 337
  if( layer->connection == NULL )
    return NULL;
338

339 340 341
  /* check if we must always create a new connection */
  close_connection = msLayerGetProcessingKey( layer, "CLOSE_CONNECTION" );
  if( close_connection && strcasecmp(close_connection,"ALWAYS") == 0 )
342
    return NULL;
343 344 345 346 347 348 349 350 351 352 353 354 355 356 357 358 359 360 361 362 363 364 365 366 367 368 369 370 371 372 373

  msAcquireLock( TLOCK_POOL );
  for( i = 0; i < connectionCount; i++ ) {
    connectionObj *conn = connections + i;

    if( layer->connectiontype == conn->connectiontype
        && strcasecmp( layer->connection, conn->connection ) == 0
        && (conn->ref_count == 0 || conn->thread_id == msGetThreadId())
        && conn->lifespan != MS_LIFE_SINGLE) {
      void *conn_handle = NULL;

      conn->ref_count++;
      conn->thread_id = msGetThreadId();
      conn->last_used = time(NULL);

      if( layer->debug ) {
        msDebug( "msConnPoolRequest(%s,%s) -> got %p\n",
                 layer->name, layer->connection, conn->conn_handle );
        conn->debug = layer->debug;
      }

      conn_handle = conn->conn_handle;

      msReleaseLock( TLOCK_POOL );
      return conn_handle;
    }
  }

  msReleaseLock( TLOCK_POOL );

  return NULL;
374 375 376 377 378 379 380 381 382 383 384 385 386 387
}

/************************************************************************/
/*                         msConnPoolRelease()                          */
/*                                                                      */
/*      Release the passed connection for the given layer.              */
/*      Internally the reference count is dropped, and the              */
/*      connection may be closed.  We assume the caller has already     */
/*      acquired the pool lock.                                         */
/************************************************************************/

void msConnPoolRelease( layerObj *layer, void *conn_handle )

{
388
  int  i;
389

390 391 392
  if( layer->debug )
    msDebug( "msConnPoolRelease(%s,%s,%p)\n",
             layer->name, layer->connection, conn_handle );
393

394 395
  if( layer->connection == NULL )
    return;
396

397 398 399
  msAcquireLock( TLOCK_POOL );
  for( i = 0; i < connectionCount; i++ ) {
    connectionObj *conn = connections + i;
400

401 402 403 404 405
    if( layer->connectiontype == conn->connectiontype
        && strcasecmp( layer->connection, conn->connection ) == 0
        && conn->conn_handle == conn_handle ) {
      conn->ref_count--;
      conn->last_used = time(NULL);
406

407 408
      if( conn->ref_count == 0 )
        conn->thread_id = 0;
409

410 411
      if( conn->ref_count == 0 && (conn->lifespan == MS_LIFE_ZEROREF || conn->lifespan == MS_LIFE_SINGLE) )
        msConnPoolClose( i );
412

413 414
      msReleaseLock( TLOCK_POOL );
      return;
415
    }
416
  }
417

418
  msReleaseLock( TLOCK_POOL );
419

420 421 422
  msDebug( "%s: Unable to find handle for layer '%s'.\n",
           "msConnPoolRelease()",
           layer->name );
423

424 425 426 427
  msSetError( MS_MISCERR,
              "Unable to find handle for layer '%s'.",
              "msConnPoolRelease()",
              layer->name );
428 429 430 431 432 433 434 435 436 437 438
}

/************************************************************************/
/*                   msConnPoolMapCloseUnreferenced()                   */
/*                                                                      */
/*      Close any unreferenced connections.                             */
/************************************************************************/

void msConnPoolCloseUnreferenced()

{
439 440 441 442 443 444 445 446 447 448 449 450 451
  int  i;

  /* this really needs to be commented out before commiting.  */
  /* msDebug( "msConnPoolCloseUnreferenced()\n" ); */

  msAcquireLock( TLOCK_POOL );
  for( i = connectionCount - 1; i >= 0; i-- ) {
    connectionObj *conn = connections + i;

    if( conn->ref_count == 0 ) {
      /* for now we don't assume the locks are re-entrant, so release */
      /* it so msConnPoolClose() can get it.  */
      msConnPoolClose( i );
452
    }
453 454
  }
  msReleaseLock( TLOCK_POOL );
455 456 457 458 459 460 461 462 463 464 465 466
}

/************************************************************************/
/*                       msConnPoolFinalCleanup()                       */
/*                                                                      */
/*      Close any remaining open connections.   This is normally        */
/*      called just before (voluntary) application termination.         */
/************************************************************************/

void msConnPoolFinalCleanup()

{
467 468
  /* this really needs to be commented out before commiting.  */
  /* msDebug( "msConnPoolFinalCleanup()\n" ); */
469

470 471 472 473
  msAcquireLock( TLOCK_POOL );
  while( connectionCount > 0 )
    msConnPoolClose( 0 );
  msReleaseLock( TLOCK_POOL );
474
}