Depends on the solution, but yes, it can happen. Take for example solution of acetic acid - initially increasing amount of acetic acid increased number of ions present in the solution, so conductivity increases. But at some point amount of water becomes too low to support further dissociation, then the dissociation shift back to the left and the conductivity goes down.
Or think about in a slightly different manner - when you mix acetic acid with water, on one end you have a pure water with its high specific resistance, on the other end you have a glacial acetic acid that is also almost not conductive, and in between you have solutions that conduct electricity quite well (classic experiment that you should be able to google).
There are several other effects that have to be taken into account, like increasing ionic strength of the solution lowering activity coefficients (and shifting dissociation equilibria) or creation of ion pairs (which effectively removes charge carries from the solution). These are dependent on the concentration.