[DCOM-288] Doctrine uses = operator instead of <= Created: 29/Jun/15  Updated: 03/Jul/15

Status: Open
Project: Doctrine Common
Component/s: Collections
Affects Version/s: 2.5.0
Fix Version/s: None

Type: Bug Priority: Minor
Reporter: Mike Assignee: Benjamin Eberlei
Resolution: Unresolved Votes: 0
Labels: None
Environment:

Windows 7.0 for dev, MySQL5.6



 Description   

I'm trying to force Doctrine to use LTE operator on activeFrom field (datetime field), but instead EQ operator is used while matching() the many-to-many relation:

    /**
     * @return SubjectInterface[]
     */
    public function getSubjects()
    {
        $expr = Criteria::expr();

        return $this->subjects->matching(
            Criteria::create()
                ->where($expr->eq('deleted', 0))
                ->andWhere($expr->eq('active', 1))
                // used date() here instead of DateTime, because I get errors with it
                ->andWhere($expr->lte('activeFrom', date('Y-m-d H:i:s'))
            )
        );
    }

creates following SQL query:

SELECT te.id AS id, te.name AS name, te.active AS active, te.activeFrom AS activeFrom, te.position AS position, te.shortDescription AS shortDescription, te.description AS description, te.deleted AS deleted, te.dateCreated AS dateCreated, te.hasChat AS hasChat, te.owner_id AS owner_id FROM subject te JOIN subject_users t ON t.subject_id = te.id WHERE t.user_id = '6' AND te.deleted = '0' AND te.active = '1' AND te.activeFrom = '2015-06-29 13:50:53'

Doctrine schema validator says my mappings are ok.

// user entity

    /**
     * @var Subject[]|ArrayCollection
     * @ORM\ManyToMany(targetEntity="Subject", inversedBy="users", indexBy="id")
     * @ORM\JoinTable(name="subject_users")
     * @ORM\OrderBy({"name"="asc"})
     */
    protected $subjects;

// subject entity

    /**
     * @var \DateTime
     * @ORM\Column(type="datetime")
     */
    protected $activeFrom;

    /**
     * @var User[]|ArrayCollection
     * @ORM\ManyToMany(targetEntity="User", mappedBy="subjects", fetch="EXTRA_LAZY", indexBy="id")
     * @ORM\OrderBy({"surname" = "ASC"})
     * @ORM\JoinTable(name="subject_users",
     *     joinColumns={@ORM\JoinColumn(name="subject_id", referencedColumnName="id")},
     *     inverseJoinColumns={@ORM\JoinColumn(name="user_id", referencedColumnName="id")}
     * )
     */
    protected $users;


 Comments   
Comment by Mike [ 30/Jun/15 ]

I wonder, if you could please tell me where query generation takes place I could debug what's happening under the hood. Probably I am not doing something right.





[DCOM-284] No Way to Get Cached File Creation Time? Created: 26/Apr/15  Updated: 26/Apr/15

Status: Open
Project: Doctrine Common
Component/s: Caching
Affects Version/s: None
Fix Version/s: None

Type: New Feature Priority: Minor
Reporter: Chad Sikorra Assignee: Benjamin Eberlei
Resolution: Unresolved Votes: 0
Labels: None


 Description   

I want to use the FilesystemCache type to store an object that gets created by parsing a large chunk of YAML. If the YAML modifcation time is greater than the cache file creation time, then I refresh the cache. However, I cannot seem to find any method that will let me actually get this information (the time that the cache file was created).

I could just use the getFilename() method of the FileCache class then use PHP functions to get the creation time, but the method is marked as protected. Is that just to keep it inline with only the methods defined on the interface? It would be nice to be able to somehow get this information.






[DCOM-253] Support use statements for annotations property type Created: 20/Sep/14  Updated: 20/Sep/14

Status: Open
Project: Doctrine Common
Component/s: Annotations
Affects Version/s: 2.4.1
Fix Version/s: None

Type: Improvement Priority: Minor
Reporter: Jáchym Toušek Assignee: Benjamin Eberlei
Resolution: Unresolved Votes: 0
Labels: None


 Description   

For example in doctrine/orm there is this annotation class:

namespace Doctrine\ORM\Mapping;

/**
 * @Annotation
 * @Target({"PROPERTY","ANNOTATION"})
 */
final class JoinTable implements Annotation
{
    /**
     * @var string
     */
    public $name;

    /**
     * @var string
     */
    public $schema;

    /**
     * @var array<\Doctrine\ORM\Mapping\JoinColumn>
     */
    public $joinColumns = array();

    /**
     * @var array<\Doctrine\ORM\Mapping\JoinColumn>
     */
    public $inverseJoinColumns = array();
}

Note the @var array<\Doctrine\ORM\Mapping\JoinColumn>. In my opinion @var array<JoinColumn> should be enough here as JoinColumn is in the same namespace. Use statements should also be supported.






[DCOM-249] Criteria are unable to locate getters for properties with an underscore prefix Created: 30/Jul/14  Updated: 14/Apr/15

Status: Open
Project: Doctrine Common
Component/s: Collections
Affects Version/s: 2.4.2
Fix Version/s: None

Type: Bug Priority: Minor
Reporter: Zach Garwood Assignee: Benjamin Eberlei
Resolution: Unresolved Votes: 0
Labels: collection, criteria, getter

Issue Links:
Reference
relates to DDC-2838 Leaky abstraction when applying Crite... Awaiting Feedback
is referenced by DCOM-275 Collections\Expr\ClosureExpressionVis... Open

 Description   

There are two related entities:
1) Inquiry:

use Doctrine\Common\Collections\ArrayCollection;
use Doctrine\Common\Collections\Criteria;
use Doctrine\ORM\Mapping as ORM;

/**
 * @ORM\Entity
 * @ORM\Table(name="inquiries")
 */
class Inquiry
{
    /**
     * @ORM\OneToMany(targetEntity="Field", mappedBy="_inquiry")
     */
    protected $_fields;

    public function __construct()
    {
        $this->_fields = new ArrayCollection();
    }

    // ...

    public function getFields()
    {
        $criteria = Criteria::create()->orderBy(['index' => Criteria::ASC]);

        return $this->_fields->matching($criteria);
    }
}

2) Field:

use Doctrine\Common\Collections\ArrayCollection;
use Doctrine\ORM\Mapping as ORM;

/**
 * @ORM\Entity
 * @ORM\Table(name="fields")
 */
class Field
{
    /**
     * @ORM\ManyToOne(targetEntity="Inquiry", inversedBy="_fields")
     * @ORM\JoinColumn(
     *      name="inquiry_id",
     *      referencedColumnName="id",
     *      nullable=false
     * )
     */
    protected $_inquiry;

    /**
     * @ORM\Column(name="index", type="integer", nullable=false)
     */
    protected $_index;

    // ...

    public function setIndex($index)
    {
        $this->_index = $index;

        return $this;
    }

    public function getIndex()
    {
        return $this->_index;
    }
}

Obviously, the underscore-prefixed property names are not ideal, but I didn't get to set the coding standard, I just have to follow it.

When trying to filter an ArrayCollection of Field entities in the Inquiry entity with an ordering criteria, I receive an ORMException with the message "Unrecognized field: index". If I alter the criteria to the following, I receive a PHP error with the message "PHP Fatal error: Cannot access protected property Field::$_index":

$criteria = Criteria::create()->orderBy(['_index' => Criteria::ASC]);

For whatever reason, Criteria::orderBy() is not finding the Field::getIndex() getter.

There are two workarounds that I've discovered for this issue:
The first is to make Field::$_index a public property:

    /**
     * @ORM\Column(name="index", type="integer", nullable=false)
     */
    public $_index;

The second, and the method that I've used, is to make an additional getter:

    public function get_index()
    {
        return $this->getIndex();
    }

I'm not sure why Criteria::orderBy() is unable to locate the Field::getIndex() getter but is able to locate Field::get_index(). And I'm sure that this is only an issue for people with ahem outdated coding standards. But it seems there is a bug with the way that the criteria are accessing getters and properties.



 Comments   
Comment by Simon Paridon [ 15/Dec/14 ]

Column names starting with an underscore seems to be a fringe case; however, this seems to be a special case of the more general problem described in DDC-2838.





[DCOM-238] FileCache does not remove directories on flush Created: 21/Mar/14  Updated: 21/Mar/14

Status: Open
Project: Doctrine Common
Component/s: Caching
Affects Version/s: None
Fix Version/s: None

Type: Bug Priority: Minor
Reporter: Ryan Korczykowski Assignee: Benjamin Eberlei
Resolution: Unresolved Votes: 0
Labels: None
Environment:

Windows 7



 Description   

FileCache::doFlush() does not remove the directories that store the cache file. The cache file is correctly deleted. This results in hundreds of hanging directories when the cache is flushed / rebuilt.

The version of doctrine/cache is 1.3.0.



 Comments   
Comment by Marco Pivetta [ 21/Mar/14 ]

This is likely because of the iterator skipping directories:

https://github.com/doctrine/cache/blob/36c4eee5051629524389da376ba270f15765e49f/lib/Doctrine/Common/Cache/FileCache.php#L120-L122





[DCOM-214] Collection Criteria isNull does not work well Created: 12/Sep/13  Updated: 12/Sep/13

Status: Open
Project: Doctrine Common
Component/s: Collections
Affects Version/s: 2.4.1
Fix Version/s: None

Type: Bug Priority: Minor
Reporter: Jan Pecek Assignee: Benjamin Eberlei
Resolution: Unresolved Votes: 0
Labels: collection, criteria, expression
Environment:

PHP 5.4.4, PostgreSQL 9.1



 Description   

I'm using the Collection project https://github.com/doctrine/collections and its Criteria building part. I'm not sure if I'm putting this bug to the good project. Please, correct me if not.

There is bug in method isNull: https://github.com/doctrine/collections/blob/master/lib/Doctrine/Common/Collections/ExpressionBuilder.php#L125

This produces SQL command where is 'IS ?' and null value is passed as param. This is fail.

Code example:

  $expr = Criteria::expr();
  $criteria = Criteria::create()
	->where($expr->lte('num', 42))
	->andWhere($expr->isNull('deleted'));
  return $this->myCollection->matching($criteria);

This produces SQL:

 SELECT ... FROM ... WHERE num <= ? AND deleted IS ? 

with params

[ 42, null ]





[DCOM-189] Doctrine Proxies may conflict with interfaced constructors Created: 03/May/13  Updated: 26/Mar/15

Status: Reopened
Project: Doctrine Common
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Improvement Priority: Minor
Reporter: Harmen M Assignee: Marco Pivetta
Resolution: Unresolved Votes: 0
Labels: None


 Description   

The Doctrine ProxyGenerator generates for a proxy a constructor. The documentation of Doctrine states that the constructor is never called. For a project, I created a group of entities with a interfaced constructor in order to enforce a common interface. This results in a incompatible proxy and so a fatal error.



 Comments   
Comment by Marco Pivetta [ 03/May/13 ]

Cannot fix this - the constructor is required to override instantiation logic

Comment by Harmen M [ 03/May/13 ]

Edit: added the correct description. I accidentially submitted the form before editing the description.

Comment by Marco Pivetta [ 03/May/13 ]

Harmen M why do you have a constructor in an interface? That's a very bad practice, and it makes things quite hard to handle.

I can think of a workaround, but I first want to be sure there's a real advantage in changing the current implementation to use

unserialize()

just to handle this specific use case.

Comment by Benjamin Eberlei [ 03/May/13 ]

Adding __construct to an interface is an anti pattern and shouldn't be done.

Comment by Harmen M [ 03/May/13 ]

Ok, then I change my implementation.

But, maybe it is an idea to update the documentation of the ORM and state that constructor interfacing is not possible?
http://docs.doctrine-project.org/projects/doctrine-orm/en/latest/reference/architecture.html

Comment by Marco Pivetta [ 18/Dec/13 ]

I actually had more requests for this feature on a similar project ( https://github.com/Ocramius/ProxyManager/issues/115 ).
I will mark this as feature request, but can't guarantee that it will get into Doctrine 2.x, since it may be a BC break.

Comment by Guilherme Blanco [ 24/Apr/14 ]

Creating interfaces with __construct ties your contract preventing extensibility points. This is nature of OOP and I do not consider this should be documented because it's (in-theory) expected that people have some level of knowledge in OO design when coding.

Closing this as invalid.

Comment by Marco Pivetta [ 24/Apr/14 ]

Re-opening.

While interfaced constructors are a known bad practice, changing constructor parameters is also a well known LSP violation (a minor one).

I'll keep tracking this, but I'm blocked by HHVM's missing Closure::bind() as of https://github.com/facebook/hhvm/issues/1203

Comment by Marco Pivetta [ 26/Mar/15 ]

Note: won't be solved in 2.5.0.

As a side-note, this was solved in ProxyManager meanwhile, but we can consider this change only for 3.0.x





[DCOM-130] Paths in Doctrine\Common\Cache\FileCache could create large directory indexes Created: 23/Oct/12  Updated: 15/Jul/15

Status: Awaiting Feedback
Project: Doctrine Common
Component/s: Caching
Affects Version/s: 2.3
Fix Version/s: None

Type: Bug Priority: Minor
Reporter: R Churchill Assignee: Benjamin Eberlei
Resolution: Unresolved Votes: 0
Labels: None
Environment:

Any



 Description   

The way paths are created within FileCache currently, there is a theoretical maximum of 16^12 directories in the cache directory, which is quite a large number. Usually schemes like this are used to restrict the number of files in one directory.

Comparing with git, for example, the dirs are arranged

00/
1c/
...
ff/

and then the object store within those directories, which is a lot more manageable, say if you happen to type ls in the cache directory, you will get a maximum listing of 256 dirs. PhpThumb does something similar when caching images.

How about something like this for getFilename():

$idHash = md5($id);
$path = substr($idHash, 0, 2) . DIRECTORY_SEPARATOR . substr($idHash, 2, 2) . DIRECTORY_SEPARATOR . substr($idHash, 4);
$path = $this->directory . DIRECTORY_SEPARATOR . $path;

return $path . $id . $this->extension;

Not nearly so elegant, but I think this has better properties for the file system. Also I would be tempted to use one of the sha family hashes and not to include the $id within the filename, but perhaps this is helpful for debugging?



 Comments   
Comment by Julian Higman [ 10/May/13 ]

We hit this problem in a live system - with a lot of cached items, the number of subdirectories that FileCache creates can exceed the number that an ext3 filesystem allows in a single directory (about 32000).

After that, an attempt to cache a new item can get an error like this:

mkdir() [function.mkdir]: Too many links

Our solution was similar to that suggested:


    protected function getFilename($id) {
        $path = implode(str_split(md5($id), 2), DIRECTORY_SEPARATOR);
        $path = $this->directory . DIRECTORY_SEPARATOR . $path;
        return $path . DIRECTORY_SEPARATOR . $id . $this->extension;
    }

It splits the md5 of the item id into parts of length 2, rather than the original 12. This creates a deeply nested structure, but which won't ever exceed the limit on number of subdirectories in any one directory. It's the same subdirectory pattern used by default by Apache mod_disk_cache, as well.

Comment by Julian Higman [ 05/Jul/13 ]

After a couple of months in production, we ran into another problem with this - we reached the maximum number of inodes in the fielsystem.

The resulting errors look like this:

mkdir() [function.mkdir]: No space left on device

There is actually disk space left, but looking at the inodes shows that the limit has been hit:

-bash-3.2# df -i
Filesystem Inodes IUsed IFree IUse% Mounted on
/dev/sda1 6553600 6553600 0 100% /

The creation of directories and subdirectories can be constrained slightly by using 3 instead of 2 characters (with hex chars, that will give max of 16^3 = 4096 subdirectories per directory, still less than the ext3 limit of 32000)

$path = implode(str_split(md5($id), 2), DIRECTORY_SEPARATOR);

but ultimately the inodes will still all be used up.

The only other options are pruning the cache at intervals, or switching to a different caching strategy altogether.

Comment by Marco Pivetta [ 05/Jul/13 ]

Julian Higman I'd suggest file-based caching mechanisms are not suited for that environment. The file cache is really meant for all those environments where there's strict constraints (like shared hosting).

Comment by Loban Rahman [ 09/Jun/15 ]

It's been a couple of years that this Bug has lain idle. Now the situation is worse, since the code now has

str_split(hash('sha256', $id), 2)

Our production system also ran out of inodes. Saying that "file-based caching mechanism is not suited for that environment, and is meant for those environments with strict constraints like shared hosting" doesn't make sense, because those environments are even more likely to run out of inodes.

The proposed solution is simple and would solve this problem. Should I make a pull request?

Comment by Trent [ 15/Jul/15 ]

Yes, this is definitely worse now because of the sha256. I have a project where there are now so many directories that it can take over an hour to do a "rm -rf app/cache"

Is there a fix in the works for this? Perhaps the name of the folders need to be increased?

I don't think reducing the number of folders would be an issue would since the files final destination is named with the id itself (so there shouldn't be any conflicts)?

str_split(str_pad(substr(preg_replace('/[^0-9]/', null, hash('sha256', $id)), 0, 3), 3, '0'), 1)

The above would only take the numbers from the hash (0-9) and will create 3 sub directories (each allowing 9 folders named 0-9)





Generated at Fri Jul 31 07:24:28 UTC 2015 using JIRA 6.2.3#6260-sha1:63ef1d6dac3f4f4d7db4c1effd405ba38ccdc558.