[DBAL-968] SQL Server modifyLimitQuery broken Created: 11/Aug/14  Updated: 11/Aug/14

Status: Open
Project: Doctrine DBAL
Component/s: Drivers, Platforms
Affects Version/s: 2.5
Fix Version/s: None
Security Level: All

Type: Bug Priority: Critical
Reporter: William Schaller Assignee: Benjamin Eberlei
Resolution: Unresolved Votes: 0
Labels: paginator, sqlserver
Environment:

SQL Server



 Description   

The recent change to SQLServerPlatform.php @jaylinski:improved sqlserver 'doModifyLimitQuery' select-from pattern broke the ORM Paginator's queries on SQL server.

I investigated, and found that some of the test cases for the SQL Server platform weren't actually correct SQL. Also, there were no test cases that covered what the paginator is doing, so I've written test cases for those. I will open a pull request for this issue.

The modifyLimitQuery method in SQLServerPlatform.php should be fixed to pass the fixed old tests and the new tests.

My concern is that that method is becoming too complex, but that's an issue for another day.



 Comments   
Comment by Marco Pivetta [ 11/Aug/14 ]

I'm gonna cry. Thank you, MSSQL, you make our lives so much "easier"





[DBAL-1057] Connection is not lazy anymore when guessing the platform is necessary Created: 05/Dec/14  Updated: 09/Dec/14

Status: Open
Project: Doctrine DBAL
Component/s: Drivers, Platforms
Affects Version/s: 2.5
Fix Version/s: None
Security Level: All

Type: Bug Priority: Critical
Reporter: Christophe Coevoet Assignee: Benjamin Eberlei
Resolution: Unresolved Votes: 4
Labels: None

Issue Links:
Duplicate
is duplicated by DBAL-1067 mysql: selecting db issue Resolved

 Description   

In DBAL 2.5, many driver can rely on different versions of the platform. Unless the version is explicitly provided, the driver will guess it at instantiation time, killing the lazyness of the connection.
This is a critical issue in any context using DI as it means that injecting the connection into anything else will connect to the server.



 Comments   
Comment by Christophe Coevoet [ 05/Dec/14 ]

Actually, the Connection class itself defers the guessing until the first time the platform is accessed. But many places in DBAL and in the ORM are retrieving the platform and storing it in a property of the class at instantiation time to avoid method calls when they need to access the platform. So this might be much harder to fix

Comment by Steve Müller [ 05/Dec/14 ]

Christophe Coevoet Can we analyze the use cases where retrieving the platform is necessary before actually connecting? I only know the custom type registering so far... Maybe we can defer that somehow?

Comment by Christophe Coevoet [ 05/Dec/14 ]

Steve Müller the issue is that many places in DBAL and the ORM are retrieving the connection before they use it. the Connection class itself in DBAL is already deferring the guessing

Comment by Steve Müller [ 05/Dec/14 ]

Christophe Coevoet I know that. The question is WHY do those defered connections need to access Connection::getDatabasePlatform() without connecting? What are the use cases?

Comment by Christophe Coevoet [ 05/Dec/14 ]

Steve Müller The issue is that all those objects are calling $connection->getDatabasePlatform() in their own constructor to store a reference to it for faster access later (no more method calls). This means that *instantiating* the ORM (or some parts of DBAL) triggers the platform guessing, which connects to the DB. This breaks the lazyness and hurts DI contexts (maybe the ORM will not even be used in this process, but it was instantiated because of being a dependency in a complex object graph).

Comment by Christophe Coevoet [ 05/Dec/14 ]

and the issue is precisely that all these parts of Doctrine are *not* deferring the retrieval of the platform.

Comment by Steffen Brem [ 07/Dec/14 ]

This is causing a lot of issues when using CI servers. Where it is very important that those things are lazy, since you do not have the database configured on most applications that build on a CI server.





[DBAL-818] Fetching identity value from an insert fails with merge replication enabled Created: 20/Feb/14  Updated: 20/Feb/14

Status: Open
Project: Doctrine DBAL
Component/s: Drivers
Affects Version/s: 2.4.2
Fix Version/s: None
Security Level: All

Type: Bug Priority: Major
Reporter: Michael Anthon Assignee: Benjamin Eberlei
Resolution: Unresolved Votes: 0
Labels: None


 Description   

The way that the PDOSqlsrv driver fetches the identity value for a freshly inserted record will fail if there are triggers on a table that do a secondary insert on a table that also has an identity column.

This is the case when you set up merge replication in SQL Server. The replication creates a series of triggers on the tables to catch any of the changes made for the purposes of replication and inserts those changes elsewhere.

We have switched to using the native SQLSrv drivers instead to work around this since that uses "SELECT SCOPE_IDENTITY() AS LastInsertId" to fetch the value during the insert command.



 Comments   
Comment by Steve Müller [ 20/Feb/14 ]

I'm not quite sure whether we can fix this is in a reasonable way. See PDO's lastInsertId method in itself is documented to be very inconsistent and behaves differently throughout different drivers and even database versions.

This method may not return a meaningful or consistent result across different PDO drivers, because the underlying database may not even support the notion of auto-increment fields or sequences.

Because of this we do not have tests in our testsuite yet that cover the last insert id topic that work as expected on all drivers.
IIRC the fact that DBAL's implementation for the native sqlsrv driver uses "SELECT SCOPE_IDENTITY() AS LastInsertId" is a workaround to support this feature at all and come around the driver limitation IIRC. This as such is a dirty workaround and should IMO not be relied on for scenarios such as you describe.
I'm not quite sure what the expected behaviour of PDO drivers in general would be concerning triggers on PK columns that do other inserts regarding last insert IDs. IMO this is not a Doctrine bug but rather an unsupported use case or even a driver bug/limitation? Not sure on this.
The only thing we could actually do is implement the same workaround we have in the native sqlsrv driver for the PDO driver. But I would rather not do this for such an edge case scenarion.
But that's just my opinion

Comment by Michael Anthon [ 20/Feb/14 ]

Yes, I agree that any workaround will be a bit of a dirty hack.

The main problem is the PDO driver using @@identity to get the last inserted id, which is pretty much the wrong way to do it in all but the simplest of cases... SCOPE_IDENTITY is there for a reason but won't work when called subsequently since it's run inside an sp_prepexec and will be out of scope anyway (it has to be tacked onto the end of the insert statement as it is in the sqlsrv driver)

There's another piece of code in the SQLSrvConnection that uses this method...

 $sql = "SELECT IDENT_CURRENT(".$this->quote($name).") AS LastInsertId";

That could potentially be used as well and would probably give a more accurate answer but is also subject to race conditions on busy systems.





[DBAL-125] Add Informix Support Created: 08/Jun/11  Updated: 11/Jun/11

Status: Open
Project: Doctrine DBAL
Component/s: Drivers
Affects Version/s: None
Fix Version/s: None

Type: New Feature Priority: Major
Reporter: Adolfo Flores Assignee: Benjamin Eberlei
Resolution: Unresolved Votes: 0
Labels: None
Environment:

Linux, Unix & Windows



 Description   

Today exist many customers with Informix 11.50 and they will deveoplment with php+simphony, but doctrine will be an alternative inside the framework but it isn't supported.
My customers can't migration ( they are happy with informix) your application and he need a framework as simphony and an abstration of your environment as Doctrine.
There is a possibility to support Informix in Doctrine?
Thanks and regards

Adolfo Flores
Consultant



 Comments   
Comment by Benjamin Eberlei [ 08/Jun/11 ]

There is a possibility to support Informix, this is described here:

http://www.doctrine-project.org/docs/dbal/2.0/en/reference/supporting-other-databases.html

I don't know informix and never worked with it, if you would volunteer to code this i would be enternelly grateful I could help you with the details of implementation.

I think a working prototype can be implemented in 4-6h of time.

Comment by Adolfo Flores [ 08/Jun/11 ]

good idea, I am an expert in informix database and I may need support in doctrine.

Is there any template to follow? I have read the required structure in the url given, it seems feasible.

What next step?

I am installing Informix, php 5.3 on RHES 5.2 All 32-bit

Regards,

Adolfo

Note: .I have no time basis, so that one week would be a reasonable time for this activity

Comment by Benjamin Eberlei [ 11/Jun/11 ]

I added some more details to the chapter, here is the part:

Implementation Steps in Detail
------------------------------

1. Add your driver shortcut to class-name `Doctrine\DBAL\DriverManager`.
2. Make a copy of tests/dbproperties.xml.dev and adjust the values to your driver shortcut and testdatabase.
3. Create three new classes implementing ``\Doctrine\DBAL\Driver\Driver``, ``\Doctrine\DBAL\Driver\Statement``
and ``Doctrine\DBAL\Driver``. You can take a look at the ``Doctrine\DBAL\Driver\OCI8`` driver.
4. You can run the testsuite of your new database driver by calling "cd tests/ && phpunit -c myconfig.xml Doctrine/Tess/AllTests.php"
5. Start implementing AbstractPlatform and AbstractSchemaManager. Other implementations should serve as good example.

Essentially Driver, Statement are the PDO equivalents of "PDO" and "PDOStatement", they even have the exact same API. OCI8 is a good example to see how it works. If Informix works with PDO then its even simpler, take a look at Doctrine\DBAL\Driver\PDO namespace, you don't need to implement the connection and statement then, just the driver taking care of the configuraiton.

The most complicated thing is to get the Platform and SchemaManager to work. Both are responsible for SQL generation for DDL statements for example. It makes sense to run the phpunit testsuite and fix the bugs when occuring until all tests pass.

Also looking at the platform and schema managers of other databases is a very good help





[DBAL-124] OCI8 Adapter (convertPositionalToNamedPlaceholders) Statemachine doesn't consider comments and strings Created: 20/May/11  Updated: 28/Jun/11

Status: Open
Project: Doctrine DBAL
Component/s: Drivers
Affects Version/s: 2.0.5
Fix Version/s: None

Type: Improvement Priority: Major
Reporter: Carsten Hetzel Assignee: Benjamin Eberlei
Resolution: Unresolved Votes: 0
Labels: None


 Description   

SQL statements may contain question marks in strings, inline comments or comment blocks. The current implementation of the method "convertPositionalToNamedPlaceholders()" doesn't consider these implications and falsely replaces them by named bind variables.

Replacement code with example:

<?php
$s = '-- Testkomm?ntar
	select	/* ? *//* ??? */ ?||\'H"al?l"o?\' as "h?""llo" union /* "? Kommentar" \' */
	/* ?" */ select \'/*Hallo\'||to_char( ? ) union select \'--Welt\'
	union--
	select ?';

echo 'In: ' . $s . PHP_EOL;

$bind = 0;
$skip = array( '--' => PHP_EOL, '/*' => '*/', '"' => '"', "'" => "'" );
for( $i = 0; $i < strlen( $s ) /* size of string might change! */; /* yes, no increment here! */ )
{
	// Skipping comments and literals
	foreach( $skip as $begin => $end )
	{
		$matches = substr_compare( $s, $begin, $i, strlen( $begin ) );
		if( $matches !== false && $matches == 0 )
		{
			$pos = strpos( $s, $end, $i+strlen( $begin ) );
			// echo "Found $begin, skipping at $i to $end at $pos" . PHP_EOL;
			if( $pos === false )
			{
				// No more data or illegal statement - anyway: no more replacements!
				// echo "EOD" . PHP_EOL;
				break 2;
			}
			$i = $pos + strlen( $end );
			continue 2; // Ensure we match /*..*//*..*/, '''' or """" - that's why we don't ++$i in the for-loop!
		}
	}
	if( $s[$i] == "?" )
	{
		// Positional to named
		// echo "Replace $bind" . PHP_EOL;
		$r = ':name' . ++$bind;
		$s = substr_replace( $s, $r, $i, 1 );
		$i += strlen( $r );
	}
	++$i;
}

echo 'Out: ' . $s . PHP_EOL;


 Comments   
Comment by Benjamin Eberlei [ 19/Jun/11 ]

This algorithmus is painfully slow. There has to be something better, why do you need to foreach the loop inside the for?

Comment by Carsten Hetzel [ 28/Jun/11 ]

Ok, sent you a PM but I'll put my response here, too:

"Painfully slow" at what circumstances? Benchmarks?

This was meant as an example implementation to show the problems of the current implementation - there's always room for improvement.

And never forget: Premature optimization is the root of all evil! ;-p

You want speed - try this one:

<?php
$sql = '-- Testkomm?ntar
        select  /* ? *//* ??? */ ?||\'H"al?l"o?\' as "h?""llo" union /* "? Kommentar" \' */
        /* ?" */ select \'/*Hallo\'||to_char( ? ) union select \'--Welt\'
        union--
        select ?';

echo 'In: ' . $sql . PHP_EOL;

$map = array();
for( $cnt = 0; $cnt < 1000; ++$cnt )
{
        $bind = 0;
        $s = $sql;

        if( isset( $map[$s] ) )
                continue;

        $strlen = strlen( $s );
        for( $i = 0; $i < $strlen /* size of string might change! */; /* yes, no increment here! */ )
        {
                $c1 = $s[$i];
                $c2 = isset( $s[$i+1] ) ? $s[$i+1] : '';
                if( $c1 == '-' && $c2 == '-')
                {
                        $pos = strpos( $s, PHP_EOL, $i+2 );
                        if( $pos === false )
                        {
                                break 2;
                        }
                        $i = $pos + 2;
                }
                elseif( $c1 == '/' && $c2 == '*' )
                {
                        $pos = strpos( $s, '*/', $i+2 );
                        if( $pos === false )
                        {
                                break 2;
                        }
                        $i = $pos + 2;
                }
                elseif( $c1 == '"' )
                {
                        $pos = strpos( $s, '"', $i+1 );
                        if( $pos === false )
                        {
                                break 2;
                        }
                        $i = $pos + 1;
                }
                elseif( $c1 == "'" )
                {
                        $pos = strpos( $s, "'", $i+1 );
                        if( $pos === false )
                        {
                                break 2;
                        }
                        $i = $pos + 1;
                }
                elseif( $c1 == "?" )
                {
                        // Positional to named
                        // echo "Replace $bind" . PHP_EOL;
                        $r = ':name' . ++$bind;
                        $s = substr_replace( $s, $r, $i, 1 );
                        $rLen = strlen( $r );
                        $i += $rLen;
                        $strlen += $rLen - 1;
                }
                else
                        ++$i;
        }
        $map[$sql] = $s;
}
echo 'Out: ' . $map[$sql] . PHP_EOL;

ATTENTION: Each "break 2;" means, you have an invalid statement! Do, whatever the statement class is supposed to do in this case!

Regards, Carsten





[DBAL-955] No exception thrown for query error Created: 31/Jul/14  Updated: 26/Oct/14

Status: Open
Project: Doctrine DBAL
Component/s: Drivers
Affects Version/s: 2.4.2
Fix Version/s: None
Security Level: All

Type: Bug Priority: Major
Reporter: Flip Assignee: Benjamin Eberlei
Resolution: Unresolved Votes: 0
Labels: None
Environment:

SQL Server 2012



 Description   

Consider the following code:

IF OBJECT_ID('tempdb..#TestTable') IS NOT NULL DROP TABLE #TestTable

CREATE TABLE #TestTable
( 
id INT  NOT NULL IDENTITY(1,1) PRIMARY KEY, 
aDate DATETIME2(6) NULL
)

INSERT INTO #TestTable
(
aDate
) VALUES
(
'2014-07-30 08:54:23.000000'
)

SELECT *
FROM #TestTable
WHERE aDate > 2000

Error:

Msg 206, Level 16, State 2, Line 21
Operand type clash: datetime2 is incompatible with smallint

Problem: for this error no DBALexception is thrown

By the way, this does work (but does not affect problem description):

SELECT *
FROM #TestTable
WHERE aDate > '2000'


 Comments   
Comment by Marco Pivetta [ 31/Jul/14 ]

Flip your code example includes no DBAL code: could you also add the PHP wrapping around those SQL statements?

Comment by Steve Müller [ 26/Oct/14 ]

Flip ping.





[DBAL-131] Remove Static Types Created: 19/Jun/11  Updated: 04/Dec/14

Status: Open
Project: Doctrine DBAL
Component/s: Drivers
Affects Version/s: None
Fix Version/s: 2.6

Type: Improvement Priority: Major
Reporter: Benjamin Eberlei Assignee: Benjamin Eberlei
Resolution: Unresolved Votes: 0
Labels: None


 Description   

the static types should be removed and made to instances of types that can differ between platforms.

Following reasons:

1. Some vendors already convert values to PHP types, for example integers. Not converting again could save about 15% wall-time for integers for example.
2. Some vendors require different type code, this is currently handled by expensive if checks.
3. If you want to change a type for one vendor but also use another vendor in the same request, then you have a problem currently.
4. If types were platform aware then we could make registering custom types simpler.






[DBAL-162] BeforeInsertID & AfterInsertID emulation workflow, wie bei MDB2 Created: 10/Sep/11  Updated: 04/Dec/14

Status: Open
Project: Doctrine DBAL
Component/s: Drivers
Affects Version/s: 2.1
Fix Version/s: 2.6
Security Level: All

Type: Improvement Priority: Major
Reporter: Benjamin Eberlei Assignee: Benjamin Eberlei
Resolution: Unresolved Votes: 0
Labels: None


 Description   

The following workflow is necessary for portable code to work on all platforms:

$nextId = $conn->nextSequenceId();
$query = "INSERT INTO (id) VALUES (?)";
$conn->executeUpdate($query, array($nextId));
$nextId = $conn->lastInsertId();





[DBAL-1050] [GH-729] Support for database URLs Created: 26/Nov/14  Updated: 05/Dec/14

Status: Open
Project: Doctrine DBAL
Component/s: Drivers
Affects Version/s: None
Fix Version/s: 2.6
Security Level: All

Type: New Feature Priority: Major
Reporter: Doctrine Bot Assignee: Benjamin Eberlei
Resolution: Unresolved Votes: 0
Labels: drivermanager, dsn, uri


 Description   

This issue is created automatically through a Github pull request on behalf of dzuelke:

Url: https://github.com/doctrine/dbal/pull/729

Message:

With a bunch of tests.

Of note:

1. in the case of information present in both URL and "normal" parameters, I'm currently giving priority to the information from the URL; IMO this makes more sense than vice versa (base info would be defined in params, and each developer or environment has a URL that may or may not override that base info, such as charset)
1. the syntax for SQLite is `sqlite:///relativepath.db` or `sqlite://ignoredhost/relativepath.db` for relative, and `sqlite:////tmp/absolutepath.db` or `sqlite://ignoredhost//tmp/absolutepath.db` for absolute paths to the database file (I went back and forth on this, but this way is easier and more consistent; https://github.com/kennethreitz/dj-database-url does the same)
1. extra query params are simply "copied" into `$params` verbatim; makes sense IMO especially considering point number 1



 Comments   
Comment by Doctrine Bot [ 04/Dec/14 ]

A related Github Pull-Request [GH-729] was closed:
https://github.com/doctrine/dbal/pull/729





[DBAL-1073] [GH-742] Take care about mariadb platform Created: 12/Dec/14  Updated: 13/Dec/14

Status: Open
Project: Doctrine DBAL
Component/s: Drivers, Platforms
Affects Version/s: 2.5
Fix Version/s: 2.5.1
Security Level: All

Type: Bug Priority: Major
Reporter: Doctrine Bot Assignee: Steve Müller
Resolution: Unresolved Votes: 0
Labels: detection, mariadb, mysql, platform, version


 Description   

This issue is created automatically through a Github pull request on behalf of nlegoff:

Url: https://github.com/doctrine/dbal/pull/742

Message:

Hi,
After upgrading to DBAL 2.5, I got an issue where I could not rename index while migrating because of MariaDB [versioning](http://en.wikipedia.org/wiki/MariaDB#Versioning) which outputs ```10.0.15-MariaDB-1~wheezy ``` as server version.

Because 10.x > 5.7 it loads new features from mysql 5.7 which are not available in mariadb ..



 Comments   
Comment by Doctrine Bot [ 13/Dec/14 ]

A related Github Pull-Request [GH-742] was assigned:
https://github.com/doctrine/dbal/pull/742





[DBAL-761] Driver\ResultStatement::fetchAll() returns empty array on a seemingly valid Driver\PDOStatement object Created: 03/Jan/14  Updated: 08/Jan/14

Status: Open
Project: Doctrine DBAL
Component/s: Drivers
Affects Version/s: 2.4
Fix Version/s: None

Type: Improvement Priority: Minor
Reporter: Dennis Matveyev Assignee: Benjamin Eberlei
Resolution: Unresolved Votes: 0
Labels: mysql, orm, query, schematool
Environment:

Windows 7 64 bit, Zend Server/Studio, PHP 5.4.16,
Server: Apache/2.2.22 (Win32) mod_ssl/2.2.22 OpenSSL/0.9.8x



 Description   

I came across a weird issue, where when running:

vendor/bin/doctrine-module orm:schema-tool:update

I would get:

[Doctrine\DBAL\Schema\SchemaException]
There is no column with name 'resource_id' on table 'role_resource'.

But I did have a column with the above name in the above table, so that was a weird message for me. So I traced it all the way to this line of code:

https://github.com/doctrine/dbal/blob/2.4/lib/Doctrine/DBAL/Connection.php#L630

If I remove "->fetchAll()" from that line, I get this object:

object(Doctrine\DBAL\Driver\PDOStatement)#531 (1)

{ ["queryString"]=> string(332) "SELECT COLUMN_NAME AS Field, COLUMN_TYPE AS Type, IS_NULLABLE AS `Null`, COLUMN_KEY AS `Key`, COLUMN_DEFAULT AS `Default`, EXTRA AS Extra, COLUMN_COMMENT AS Comment, CHARACTER_SET_NAME AS CharacterSet, COLLATION_NAME AS CollactionName FROM information_schema.COLUMNS WHERE TABLE_SCHEMA = 'loginauth' AND TABLE_NAME = 'role_resource'" }

Which has a valid SQL query that runs fine and shows Field names on my local machine's MySQL prompt. But when I add ->fetchAll() back in that line of code, an empty array is returned, field names are not returned, and a SchemaException is generated. I found this odd and wanted to report it. Whether it is a bug or not, hopefully I can find the cause of this issue.

For loads more info, please see this: http://stackoverflow.com/questions/20907491/doctrine-2-there-is-no-column-with-name-columnname-on-table-table



 Comments   
Comment by Steve Müller [ 03/Jan/14 ]

Dennis Matveyev It seems you are using MySQL. Can you please tell which version you use?

Comment by Steve Müller [ 03/Jan/14 ]

Dennis Matveyev Please also provide whether you use mysqli or PDO_MYSQL.

Comment by Steve Müller [ 03/Jan/14 ]

Dennis Matveyev Sounds dumb, but also are you sure that the vendor/bin/doctrine-module orm:schema-tool:update uses the correct connection parameters and does not by error connect to another database server or something? Because otherwise everything looks good IMO.

Comment by Dennis Matveyev [ 07/Jan/14 ]

Yes, I am using:
Server version: 5.5.23-log MySQL Community Server (GPL)
mysql client Ver 14.14 Distrib 5.5.23, for Win32 (x86)

Doctrine connection string for 'driverClass' is 'Doctrine\DBAL\Driver\PDOMySql\Driver', so it is PDO_MYSQL.

I am connecting to the right database, but your last suggestion is what helped to uncover the issue. When connecting to the database with the very same user/pass, and running the command, here is what I saw:

mysql> select resource_id from role_resource;
ERROR 1142 (42000): SELECT command denied to user 'login'@'localhost' for table 'role_resource'

Running a GRANT command to allow SELECT for this user solved the problem.

To improve debugging of similar issues, I'd see if there is a way to propagate the error from MySQL server to ./doctrine-module,
or
improve the error message of SchemaException, to i.e. "There is no column with name 'action_id' on table 'role_action', or database permissions prevent table access."

Thanks!

Comment by Steve Müller [ 07/Jan/14 ]

Dennis Matveyev Thank you for reporting this and I'm glad I could help. I will mark this as improvement, though. You are right in that the root cause of the error should be propagated to the user instead.

Comment by Steve Müller [ 08/Jan/14 ]

Dennis Matveyev Okay I found the root cause of the problem. See what MySQL states about access to the information_schema table for retrieving metadata about a certain database:

Each MySQL user has the right to access these tables, but can see only the rows in the tables that correspond to objects for which the user has the proper access privileges. In some cases (for example, the ROUTINE_DEFINITION column in the INFORMATION_SCHEMA.ROUTINES table), users who have insufficient privileges will see NULL.

So it returns NULL instead of raising an error and this is why Doctrine is not able to propagate the proper exception in this case. I'm not sure if changing the exception message in the SchemaException class is the proper way of handling this. Also please note that this is not only about columns (where you got stuck at) but can occurr for almost every list*() action called in the SchemaManager I suppose. Furthermore this might be a MySQL specific issue, not sure about that.

The proper solution would be to throw an exception much earlier, to have a decent behaviour in such an edge case. I think we should throw an exception here: https://github.com/doctrine/dbal/blob/master/lib/Doctrine/DBAL/Schema/AbstractSchemaManager.php#L267 because a table cannot live without columns and indicates an error somewhere. But I would like to hear the opinion of Benjamin Eberlei on that issue.

Comment by Steve Müller [ 08/Jan/14 ]

I checked the SQL Server documentation and it seems they behave exactly the same: http://technet.microsoft.com/en-us/library/ms187113.aspx

In SQL Server 2005 and later, the visibility of metadata is limited to securables that a user either owns or on which the user has been granted some permission. For example, the following query returns a row if the user has been granted a permission such as SELECT or INSERT on the table myTable.

However, if the user does not have any permission on myTable, the query returns an empty result set.





[DBAL-858] oracle IN statement with more than 1000 values Created: 11/Jan/13  Updated: 01/Apr/14

Status: Open
Project: Doctrine DBAL
Component/s: Drivers, Platforms
Affects Version/s: 2.2.2
Fix Version/s: None
Security Level: All

Type: Improvement Priority: Minor
Reporter: Marc Drolet Assignee: Benjamin Eberlei
Resolution: Unresolved Votes: 0
Labels: None


 Description   

If I have a query with a IN statement with more tahn 1000 values I get an sql error.

I've try IN with implode:
select * from test where id IN(' . implode(',', $values) . ')
and I've also try with executeQuery:
select * from test where id IN(:test)
executeQuery($sql, array($values), array(\Doctrine\DBAL\Connection::PARAM_INT_ARRAY))



 Comments   
Comment by Marc Drolet [ 11/Jan/13 ]

Here is the way I've implement the solution on my side: (for oracle)

into Doctrine/DBAL/Statement.php, I've add this method:

/**
     * Binds a parameter value to the statement.
     * This is implemented this way for oracle only. Other drivers are redirected to bindValue method.
     *
     * The value will be bound with to the type provided (that required to be a table type).
     *
     * @param String $name The name or position of the parameter.
     * @param Array $value The value of the parameter.
     * @param String $type The name of the type to use to bind.
     * @return boolean TRUE on success, FALSE on failure.
     */
    public function bindList($name, Array $value, $type)
    {
        if ('oracle' !== $this->platform->getName())
        {
            $this->bindValue($name, $value, $type);
        }
        else
        {
            return $this->stmt->bindList($name, $value, $type);
        }
    }

into Doctrine/DBAL/Driver/Statement.php I've add:

/**
     * @TODO: docs
     */
    function bindList($param, Array $values, $type);

into Doctrine/DBAL/Driver/OCI8/OCI8Statement.php I've add this method:

/**
     * {@inheritdoc}
     */
    public function bindList($param, Array $value, $type)
    {
        if (!($list = oci_new_collection($this->_dbh, $type)))
        {
            //throw new OCI8Exception::fromErrorInfo($this->errorInfo());
        }

        foreach ($value as $entry)
        {
            $list->append($entry);
        }
        if (!oci_bind_by_name($this->_sth, $param, $list, -1, OCI_B_NTY))
        {
            //throw new OCI8Exception::fromErrorInfo($this->errorInfo());
        }
    }

// NOTE: we should probably add the bindList to all driver Statement object.

into your code you can use it this way:

$sql = "
    SELECT *
    FROM test
    WHERE id IN
    (
        SELECT *
        FROM
        (
            CAST (: p_ids AS list_int_type)
        )
    )
";
$stmt = connection->prepare($sql);
$stmt->bindList(': p_ids', $ids, 'list_int_type');
$stmt->execute();
$rs = $stmt->fetchAll(PDO::FETCH_ASSOC);

NOTE:
list_int_type need to be a valid oracle data type. You can create one with the name you want.
example:
you can have 2 type of accepted array of values: integer and string
let's say we create one for string named: list_str_type and one for integer list_int_type

create or replace type list_str_type as table of varchar2(4000);
create or replace type list_int_type as table of number;

Comment by Benjamin Eberlei [ 01/Apr/13 ]

Hey Marc Drolet

thanks for the feedback and the solution, however i would like to have something generic that is working independent of the database driver. This code is very specific.

Can you point me to some documentation why oci collection works with more than 1000 elements and how it works in PHP?

Comment by Marc Drolet [ 02/Apr/13 ]

Hi Benjamin,

The limitation is not from the oci driver, it's an oracle limitation. There are a couple of possible solution/implementation that can be done but the one I've provide is the one that perform better for the test I've done and from what I can found over the blogs I've read.

I can't find the exact documentation of oracle. oracle doc is so poor.
Here is the best description link I can provide that describe some possible implementation.
http://vsadilovskiy.wordpress.com/substituting-a-collection-for-in-list-performance-study/

I don't know if there is similar limitation with other database. With the implementation I've provided, It will be possible to implement the proper solution depending on the database limitation you face otherwise it will execute the generic IN. What's bad, we need to create the type into the database.

NOTE: In my case, I can not perform a sub-query, I get the my collection from a web service call.





[DBAL-167] Schema comparator doesn't work properly with columnDefinition's Created: 17/Sep/11  Updated: 16/Jul/12

Status: Open
Project: Doctrine DBAL
Component/s: Drivers, Platforms, Schema Managers
Affects Version/s: 2.0.8, 2.1, 2.1.1, 2.1.2
Fix Version/s: None
Security Level: All

Type: Bug Priority: Minor
Reporter: Dmitry Strygin Assignee: Benjamin Eberlei
Resolution: Unresolved Votes: 3
Labels: None


 Description   

Schema comparator will mostly always return changed properties on columns for entities defined with columnDefinition even they are identical in the DB. This is due to weak low-lever compatibility of SchemaTool#getCreateSchemaSql() and SchemaTool#getSchemaFromMetadata() – the first one doesn't reconstruct columnDefinition, and the other one never supports 'fixed', 'default', cannot determine, whether it is boolean or integer (ex. TINYINT in the DB), etc...

All this results in extremely annoying unnecessary alter-table-change-columns surrounded by dropping and after that re-enabling constrains dependent on those columns.

I mean stuff like this:

symfony2#app/console doctrine:schema:update --dump-sql
...
ALTER TABLE es_hotels DROP FOREIGN KEY FK_527F88EE584598A3F92F3E70;
ALTER TABLE es_hotels DROP FOREIGN KEY FK_527F88EE584598A37A3ABE5D;
ALTER TABLE es_hotels DROP FOREIGN KEY FK_527F88EE584598A3EE551564;
ALTER TABLE es_hotels CHANGE is_active is_active TINYINT(1) NOT NULL DEFAULT '1', CHANGE checksum checksum CHAR(32) DEFAULT NULL;
ALTER TABLE es_hotels ADD CONSTRAINT FK_527F88EE584598A3F92F3E70 FOREIGN KEY (operator_id, country_id) REFERENCES es_countries(operator_id, id) ON DELETE CASCADE;
ALTER TABLE es_hotels ADD CONSTRAINT FK_527F88EE584598A37A3ABE5D FOREIGN KEY (operator_id, resort_id) REFERENCES es_resorts(operator_id, id) ON DELETE CASCADE;
ALTER TABLE es_hotels ADD CONSTRAINT FK_527F88EE584598A3EE551564 FOREIGN KEY (operator_id, subresort_id) REFERENCES es_subresorts(operator_id, id) ON DELETE CASCADE;
...

The simple solution would be to fix schema comparator not to signal any changes on columns with columnDefinition properties.
But would be much and much better to add some code to all *SchemaManager#getPortableTableColumnDefinition so they would reconstuct columnDefinition and they would be matched in the schema comparator.

I can do this



 Comments   
Comment by Roderick Schaefer | We handle IT [ 16/Oct/11 ]

I'm having the same issue on my production webserver, but not on the development webserver. I find that odd. It tries to drop all foreign keys and create them again, although without the CHANGE statement you are referring to, Dmitry.

Comment by Benjamin Eberlei [ 09/Jan/12 ]

This maybe fixable by making a hash out of the column definition and saving it into a database comment.

The Foreign Key problem maybe because of an old MySQL version 5.0.x

Comment by Joe Cai [ 16/Jul/12 ]

@beberlei, sounds good to me. any plan of implementing this?





[DBAL-879] Sequence default value [PGSQL] Created: 29/Apr/14  Updated: 26/Oct/14

Status: Open
Project: Doctrine DBAL
Component/s: Drivers, Schema Managers
Affects Version/s: 2.4.2
Fix Version/s: None

Type: Bug Priority: Minor
Reporter: Mohammad Niknam Assignee: Benjamin Eberlei
Resolution: Unresolved Votes: 0
Labels: autoincrement, postgresql, sequence
Environment:

ArchLinux
PostgreSQL 9.3.4



 Description   

Hi
I'm using dbal to generate schmea from database via Schema-Manager. The problem is that my primary field 'id' have default value of 'nextval('test1_id_seq'::regclass)' but when I retrive columns using Doctrine\DBAL\Schema\AbstractSchemaManager::listTableDetails() or Doctrine\DBAL\Schema\Table::getColumns() , default value of the column 'id' is null.
In Doctrine\DBAL\Schema\PostgreSqlSchemaManager::_getPortableTableColumnDefinition() method at line 292 default value replaced with null, I don't know why but I guess It's because Driver compatibility.
Also Doctrine\DBAL\Schema\Sequence has no method to retrieve that table.
So I don't have the default value (pointing at sequence) and I can't find out what Sequence is linked to this table either.



 Comments   
Comment by Steve Müller [ 26/Oct/14 ]

Can you please provide a code example of how you create table + sequence and retrieve it?





[DBAL-933] Get Statement Column Metadata Created: 05/Jul/14  Updated: 06/Jul/14

Status: Open
Project: Doctrine DBAL
Component/s: Drivers
Affects Version/s: None
Fix Version/s: None

Type: New Feature Priority: Trivial
Reporter: Benoît Burnichon Assignee: Benjamin Eberlei
Resolution: Unresolved Votes: 0
Labels: None


 Description   

It would be nice to have a utility method in \Doctrine\Dbal\Driver\ResultStatement to properly retrieve the result column names.

Currently, we have to rely on somthing like
$columnNames = array_keys($statement->fetch(\PDO::FETCH_ASSOC));

Problem: It does not work with empty result-sets, and more checks should be performed to handle these.

With PDO, http://www.php.net/manual/en/pdostatement.getcolumnmeta.php could be used to properly retrieve names.

For Sqlite3 it is easy, http://www.php.net/manual/en/sqlite3result.columnname.php

For Mysql, http://www.php.net/manual/en/mysqli-result.fetch-fields.php would do the trick






[DBAL-1054] Expose native database handler from Connection Created: 03/Dec/14  Updated: 03/Dec/14

Status: Open
Project: Doctrine DBAL
Component/s: Drivers
Affects Version/s: 2.4.3
Fix Version/s: None
Security Level: All

Type: Improvement Priority: Trivial
Reporter: Davide Romanini Assignee: Benjamin Eberlei
Resolution: Unresolved Votes: 0
Labels: None


 Description   

DBAL drivers totally hide the wrapped native connection handler. Sometimes it is useful having a way to access it, eg: integrating old non-dbal code sharing the same connection, or to access specific funcionalities not available through the driver.
With oci8 for example I'd want to use oci_set_client_info/oci_set_module_name/oci_set_action on postConnect for better dba auditing and analysis. In current implementation the only way is to use reflection and access the protected $dbh instance variable, that's very fragile.
I propose to add a Connection::getNativeConnection() or similar name for this purpose and trivially implement it on the various drivers. BC compatibility could be an issue here, but eventually it could be factored in a more specific interface.



 Comments   
Comment by Steve Müller [ 03/Dec/14 ]

Davide Romanini which Connection are you refering to? The DBAL Connection class provides a getWrappedConnection() method which returns the underlying driver connection object.
If you refer to driver connections in the first place I doubt we will expose the internal connection ressource to avoid possible weird side effects. We had similar discussion several times now. If you need advanced access to driver specific functionality on the connection, please don't use DBAL. DABL is about abstraction accross different drivers and database vendors. It is not intended for advanced driver specific usage.

Comment by Davide Romanini [ 03/Dec/14 ]

getWrapperConnection() returns the DBAL specific connection (OCI8Connection in my use case). That is not a real "native" connection since it is a mere wrapper around the oci8 php module. In particular it doesn't allow to access some specific functions such as:

  • oci_set_action / oci_set_module_name / oci_set_client_info
  • oci_set_prefetch
  • ... other useful functions
    In my case I'm creating a listener that uses the first functions on the list to useful contextual values (eg: symfony controller/action names) to allow fine tuning and monitoring at the dba level. Obviously this is implemented as a cross-cutting concern with a listener without even touching the main application code. So your advise to not using DBAL for such a purpose seems impractical at best.
    Probably a cleaner solution could be to fully encapsulate all the oci8 functions in the OCI8Connection wrapper, creating a real "native" connection. But this is an hard effort, so my approach is more pragmatic.
    I really don't see the whole point of avoiding to "expose the internal connection ressource to avoid possible weird side effects". As a developer I should in any case know what I'm doing and especially when gradually migrating old oci8-based code sharing the same connection is the best approach in my experience. As a side note, improper use of a Connection::getNativeConnection() is just as harmful at least as improper use of EntityManager::getConnection().




Generated at Thu Dec 18 07:24:42 UTC 2014 using JIRA 6.2.3#6260-sha1:63ef1d6dac3f4f4d7db4c1effd405ba38ccdc558.