Efficient cone searches with Healpix-alchemy

  • Context: Graduate 
  • Thread starter Thread starter BOAS
  • Start date Start date
  • Tags Tags
    Cone Python
Click For Summary
SUMMARY

The discussion focuses on optimizing cone searches using the HEALPix-alchemy package in Python. The user has implemented a database schema with classes such as Field, FieldTile, and Source, and is utilizing CDSHealpix and MOCpy for HEALPix cell extraction. The current query method, which checks each source against FieldTile, is inefficient. The user seeks a more efficient approach to check each tile and return the contained sources.

PREREQUISITES
  • Familiarity with SQLAlchemy ORM for database modeling
  • Understanding of HEALPix and its applications in astronomy
  • Knowledge of spatial indexing techniques in databases
  • Experience with Python programming and relevant libraries such as CDSHealpix and MOCpy
NEXT STEPS
  • Research spatial indexing methods in SQLAlchemy to improve query performance
  • Learn about optimizing queries with PostGIS for spatial data
  • Explore advanced features of HEALPix-alchemy for efficient data handling
  • Investigate the use of Multi Order Coverage (MOC) for spatial data representation
USEFUL FOR

Astronomers, data scientists, and software developers working with spatial data and seeking to optimize HEALPix-based queries in their applications.

BOAS
Messages
546
Reaction score
19
TL;DR
Implementation of a more efficient cone search using healpix-alchemy
I am trying to implement a cone search for a catalog with the HEALPix-alchemy package. I have the following models set up, based on the examples here: https://arxiv.org/pdf/2112.06947


Python:
class Field(Base):
        """
        Represents a collection of FieldTiles making up the area of interest.
        """
        id = Column(Integer, primary_key=True, autoincrement=True)
        tiles = relationship(lambda: FieldTile, order_by="FieldTile.id")


class FieldTile(Base):
    """
    A HEALPix tile that is a component of the Field being selected.
    """
    id = Column(ForeignKey(Field.id), primary_key=True)
    hpx = Column(Tile, index=True)
    pk = Column(Integer, primary_key=True, autoincrement=True)


class Source(Base):
    """
    Represents a source and its location.
    """
   
    id = mapped_column(Integer, primary_key=True, index=True, autoincrement=True)
    name = Column(String, unique=True)
    Heal_Pix_Position = Column(Point, index=True, nullable=False)


I am then using CDSHealpix to get the HEALPix cells contained within a specified cone.

I construct the Multi Order Coverage map from the HEALPix cells using MOCpy and extract the HEALPix tiles using HEALPix-alchemy.

I then populate the Field table with this collection of tiles.

Finally, I perform the following query:

Python:
query = db.query(Source).filter(FieldTile.hpx.contains(Source.Heal_Pix_Position)).all()

However, this is very inefficient as I am effectively checking each source in my catalog to see if it is contained within a FieldTile.

How can I modify my approach so that I am checking each tile and returning the sources that it contains?
 

Similar threads

  • · Replies 5 ·
Replies
5
Views
3K
  • · Replies 3 ·
Replies
3
Views
3K
  • · Replies 8 ·
Replies
8
Views
2K
  • · Replies 3 ·
Replies
3
Views
2K
  • · Replies 1 ·
Replies
1
Views
2K
  • · Replies 5 ·
Replies
5
Views
3K
  • · Replies 8 ·
Replies
8
Views
1K
  • · Replies 1 ·
Replies
1
Views
2K
  • · Replies 3 ·
Replies
3
Views
3K
  • · Replies 20 ·
Replies
20
Views
10K