Fast way to search the table for two columns closest values

  • Context: Undergrad 
  • Thread starter Thread starter prayami
  • Start date Start date
  • Tags Tags
    Columns Search Table
Click For Summary
SUMMARY

The discussion focuses on efficiently finding the closest values in a table with columns Temperature, Field, Vhall, and Vdrive, based on given Vhall_measured and Vdrive_measured values. The user seeks a mathematical approach to quickly identify rows that match these criteria, considering both direct comparisons and ranges defined by Vhall_min, Vhall_max, Vdrive_min, and Vdrive_max. The consensus is that without sorted data, the fastest method remains O(n) linear search, despite attempts to utilize binary search techniques. The conversation highlights the limitations of searching unordered lists and the practicality of the problem given the dataset size of 1000 rows.

PREREQUISITES
  • Understanding of linear search algorithms
  • Familiarity with binary search principles
  • Basic knowledge of mathematical concepts related to distance metrics
  • Experience with programming for data manipulation
NEXT STEPS
  • Implement a linear search algorithm to find closest values in datasets
  • Explore mathematical distance metrics for multi-dimensional data
  • Learn about data sorting techniques to optimize search operations
  • Investigate the use of matrix operations for efficient data querying
USEFUL FOR

Data analysts, software developers, and anyone involved in optimizing search algorithms within datasets, particularly those working with numerical data in programming contexts.

prayami
Messages
1
Reaction score
0
Hi,

I got one table of four columns.
e.g.
Temperature, Field, Vhall and Vdrive.

Table contains the 1000 rows.

I am given the Vhall_maesured and Vdrive_measured values.

I want to find out, which row in the table has the closest values
for Vhall and Vdrive to corresponding Vhall_maesured and Vdrive_measured.

Is there any mathematical way to find the closest row quickly.

If we can't then, we can divide this problem further:
We can have
Vhall_min and Vhall_max
Vdrive_min and Vdrive_max

And in the table we want to find any of the row( there can be more than one
but we are fine if we find any one) where Vhall and Vdrive fall between the above min and max
values.

I know computer programming and finally I want to use it in my program. Right now I am
checking each row sequencially. But I am looking the fatest to reach that point.

I have used one another method i.e. binary search and linear search combine but I am
thiking if there any way using maths of the matrix or vectro to find the row quickest.

Thanks,
 
Physics news on Phys.org
Im not sure if you can do this because I don't program, but;

Make a third column which gives the values of the differences of the 2 other columns. Then use the (Third column)_min function.
 
There is no faster way than O(n) to search an unordered list!

You mention you are using binary search, so maybe we can assume the data is sorted (e.g. the variables Vhall and Vdrive are monotonicall increasing)?

Without additional assumptions on the data, then linear search is the fastest method. Out of curiosity why is speed even a consideration when you are considering only a 1000 data points in the year 2008? We could have found the minimum of thousands of such data sets in the time you or I spent typing these messages!
 

Similar threads

  • · Replies 0 ·
Replies
0
Views
3K
  • · Replies 2 ·
Replies
2
Views
2K
  • · Replies 7 ·
Replies
7
Views
3K
Replies
1
Views
968
  • · Replies 21 ·
Replies
21
Views
3K
  • · Replies 2 ·
Replies
2
Views
2K
  • · Replies 8 ·
Replies
8
Views
2K
Replies
1
Views
2K
  • · Replies 32 ·
2
Replies
32
Views
3K
  • · Replies 3 ·
Replies
3
Views
3K