A 1D finite square well is generally defined either by \begin{equation} V(x)=\begin{cases} 0\qquad -a\leqslant x\leqslant a\\ V_0\qquad \text{otherwise} \end{cases} \tag{1} \end{equation} or \begin{equation} V(x)=\begin{cases} -V_0\qquad -a\leqslant x\leqslant a\\ 0\qquad \text{otherwise} \end{cases} \tag{2} \end{equation}
Where $a>0$ and $V_0>0$. It is just a matter of defining the zero of your potential.
Infinite square well can be regarded as the limit $V_0\to+\infty$ of $(1)$, i.e.
\begin{equation} V(x)=\begin{cases} 0\qquad -a\leqslant x\leqslant a\\ +\infty\qquad \text{otherwise} \end{cases} \tag{3} \end{equation}
In this case the wavefunction is zero outside the well and you solve Schrödinger equation inside applying boundary conditions.
What about taking the limit of $(2)$? \begin{equation} V(x)=\begin{cases} -\infty\qquad -a\leqslant x\leqslant a\\ 0\qquad \text{otherwise} \end{cases} \tag{4} \end{equation}
Since we are now dealing with infinities I'm not sure whether saying $(3)$ and $(4)$ only differ for the choice of the of the potential. Also, what can I say about the wave function inside and outside in case $(4)$?