Points inside a rhombus in 3-dimensional sapce

[rbwang1225]

I have a lattice plane in 3 dimensional space.
Now, I want to cut it in a unit cell, but I have to write some code the make things get easier.
As you can see from my figure, I can know the four coordinates of the highlighted atoms which form the boundaries of the top plane.
My problem is if the four coordinates are known, how could I eliminate the atoms outside the boundary, or how could I know the limitations of the coordinates $(x_i,y_i,z_i)$ of atoms inside the boundary.

My idea is to use their slopes between each pair of highlighted atoms.

Any advices would be very appreciated!

 

[haruspex]

As you can see from my figure,

What figure?

 

[Bill Simpson]

if the four coordinates are known, how could I eliminate the atoms outside the boundary

This seems like a simple easy to understand method.

For each point p which you want to test to determine if it is inside the rhombus,
For each of the four verticies v making up your rhombus,
You have two edge vectors from v, call those vector1 and vector 2.
Express your point p as the unique p=const1*vector1+const2*vector2.
If const1 and const2 are greater than or equal to zero then the point lies within on on the edge of the rhombus, at least as far as that vertex v is concerned.
Think a few minutes until you can convince yourself why that is true.
If you repeat this for all four verticies and they all agree it is in or on the edge then that point p is inside your rhombus.
Repeat for all other points.

There is also a method that tests which side of a line a point lies on. If you use each edge of your rhombus as a line then that will also give the same result. But it seems easier for students to introduce a sign error somewhere when using this method than when using the previous test.