[DC-841] Doctrine_Connection_Mssql::replaceBoundParamsWithInlineValuesInQuery regex failing to replace all '?' instances [patch+] Created: 25/Aug/10  Updated: 05/Mar/11

Status: Open
Project: Doctrine 1
Component/s: Connection
Affects Version/s: 1.2.3
Fix Version/s: None

Type: Bug Priority: Major
Reporter: Daniel Cousineau Assignee: Jonathan H. Wage
Resolution: Unresolved Votes: 3
Labels: None
Environment:

PHP 5.2.11, Apache, Microsoft SQL Server 2005



 Description   

When executing queries with WHERE statements using multiple instances of the "<>" operator (as well as other non =,( symbols inbetween definitions), the method Doctrine_Connection_Mssql::replaceBoundParamsWithInlineValuesInQuery fails to identify all ? replacements.

In the following piece of code I have a query (trimmed for readability and renamed for privacy) that fails to have all "?" symbols replaced as well as the relevant code from the method mentioned above (minus the return statement) doing a simple demonstration:

<?php 
$query = "SELECT * FROM [table] AS [t] WHERE ([t].[field1] <> ? AND [t].[field2] <> ? AND [t].[field2] LIKE ?)";

$params = array(
	"'param1'",
	"'param2'",
	"'param3'"
);


/**
 * Replaces bound parameters and their placeholders with explicit values.
 *
 * Workaround for http://bugs.php.net/36561
 *
 * @param string $query
 * @param array $params
 */
//protected function replaceBoundParamsWithInlineValuesInQuery($query, array $params) {

	foreach($params as $key => $value) {
		if(is_null($value)) {
			$value = 'NULL';
		}
		else {
			//$value = $this->quote($value); //REMOVED AS PRE-ADDED QUOTES TO ABOVE PARAMETER LIST
		}

		$re = '/([=,\(][^\\\']*)(\?)/iU';
		
		$matches = array();
		preg_match($re,$query,$matches);
		
		var_dump($matches); //ADDED FOR DEMONSTRATION

		$query = preg_replace($re, "\\1 {$value}", $query, 1);
		
		var_dump($query); //ADDED FOR DEMONSTRATION
	}

//	return $query;
//
//}

Running this code produces:

array(3) {
  [0]=>
  string(18) "([t].[field1] <> ?"
  [1]=>
  string(17) "([t].[field1] <> "
  [2]=>
  string(1) "?"
}
string(108) "SELECT * FROM [table] AS [t] WHERE ([t].[field1] <>  'param1' AND [t].[field2] <> ? AND [t].[field2] LIKE ?)"
array(0) {
}
string(108) "SELECT * FROM [table] AS [t] WHERE ([t].[field1] <>  'param1' AND [t].[field2] <> ? AND [t].[field2] LIKE ?)"
array(0) {
}
string(108) "SELECT * FROM [table] AS [t] WHERE ([t].[field1] <>  'param1' AND [t].[field2] <> ? AND [t].[field2] LIKE ?)"

Unfortunately the regex will not identify all the ? instances properly in the query when run like preg_match_all(), which was my first idea to fix (pre-identify all ? instances, then go through and replace them).

The only 3 potential solutions I can think of are:

1. Pre-identify all ?'s and note their position in the string, to do this using a much looser regex, then replace all the ?'s found
2. Use a stack a loop and a switch statement to scan for eligible ? replacements. I have some sample code that shows this should work, but performance is not something I have tested.
3. Revert back to the previous behavior. Scanning the changelogs I really don't know where this broke, other than queries that worked prior to 1.2.3 have since broken and this method is the cause.



 Comments   
Comment by Daniel Cousineau [ 25/Aug/10 ]

I am probably way over thinking a solutions, however since I have to run home and don't have time to flesh this out further at the moment, my initial idea is something like this:

<?php 
$query = "SELECT * FROM [table] AS [t] WHERE ([t].[field1] <> 'Testing!?' AND [t].[field2] <> ? AND [t].[field?] LIKE ? AND [t].[field3] = ?)";

$params = array(
	"'param1'",
	"param2?",
	"'param3'"
);

var_dump($query);

$stack = array();
$stringDelim = array("'", '"');

$i = 0;
foreach( str_split($query) as $char )
{
	switch($char) {
		
		case "[":
			if( !in_array(end($stack), $stringDelim) )
				array_push($stack, $char);
			
			break;
			
		case "]":
			if( end($stack) == "[" ) {
				array_pop($stack);
			} else
				array_push($stack, $char);
				
			break;
		
		case "\"":
		case "'":
			if( end($stack) == $char )
				array_pop($stack);
			else
				array_push($stack, $char);
			
			break;
		
		case "?":
			if( !in_array(end($stack), array_merge($stringDelim, array("[", "]"))) )
			{
				$param = array_shift($params);
				$query = substr_replace($query, $param, $i, 1);
				
				$i += strlen($param) - 1;
				var_dump($query);
			}
			
			
			break;
		
		default:
	}
	
	$i++;
}

var_dump($query);

Which produces

string(131) "SELECT * FROM [table] AS [t] WHERE ([t].[field1] <> 'Testing!?' AND [t].[field2] <> ? AND [t].[field?] LIKE ? AND [t].[field3] = ?)"
string(138) "SELECT * FROM [table] AS [t] WHERE ([t].[field1] <> 'Testing!?' AND [t].[field2] <> 'param1' AND [t].[field?] LIKE ? AND [t].[field3] = ?)"
string(144) "SELECT * FROM [table] AS [t] WHERE ([t].[field1] <> 'Testing!?' AND [t].[field2] <> 'param1' AND [t].[field?] LIKE param2? AND [t].[field3] = ?)"
string(151) "SELECT * FROM [table] AS [t] WHERE ([t].[field1] <> 'Testing!?' AND [t].[field2] <> 'param1' AND [t].[field?] LIKE param2? AND [t].[field3] = 'param3')"
string(151) "SELECT * FROM [table] AS [t] WHERE ([t].[field1] <> 'Testing!?' AND [t].[field2] <> 'param1' AND [t].[field?] LIKE param2? AND [t].[field3] = 'param3')"

Which is what we would expect, as well as it doesn't take into account for newly inserted data containing '?' symbols.

But again this is just playing around with potential solutions.

Comment by Enrico Stahn [ 27/Aug/10 ]

The patch for DC-545 broke some queries or didn't worked at all. Unfortunately there was no Test Case for DC-545.

The patch and a Test Case can be found at: http://github.com/estahn/doctrine1/compare/master...DC-841

Comment by Enrico Stahn [ 02/Sep/10 ]

I made a mistake with github, the updated branch can be found at
http://github.com/estahn/doctrine1/tree/DC-841-2

Comment by Lionel ROTA [ 05/Mar/11 ]

Doesn't work with :

'Test' <> 'Test !?'

The question mark is captured...

This code seems working :

foreach($params as $key => $value) {
  if(is_null($value)) {
    $value = 'NULL';
  }
  else {
    $value = $this->quote($value);
  }

  $re = '/((?:[=<>,\(]|LIKE|IS)[^\\\']*)(\?)/iuU';

  $query = preg_replace($re, "\\1 {$value}", $query, 1);
}
Generated at Mon Dec 22 16:24:47 UTC 2014 using JIRA 6.2.3#6260-sha1:63ef1d6dac3f4f4d7db4c1effd405ba38ccdc558.