Sudoku Programmers Forum Index

 
 FAQFAQ   SearchSearch   MemberlistMemberlist   UsergroupsUsergroups   RegisterRegister   ProfileProfile   Log inLog in          Games  Calendar

Log in to check your private messagesLog in to check your private messages   

Nishio vs Supercolouring
Goto page 1, 2, 3  Next
 
Post new topic   Reply to topic    Sudoku Programmers Forum Index -> Solving sudoku
View previous topic :: View next topic  
Author Message
AMcK

Joined: 07 Apr 2005
Posts: 89
:
Location: Cambridge

Items
PostPosted: Sat Jul 30, 2005 12:55 am    Post subject: Nishio vs Supercolouring Reply with quote

Apologies for the length and detail of this post but I suspect that I may have stumbled on a major breakthrough.

I finally discovered the last rule in multi-colouring that allowed it so solve all of Nick 70’s advanced colouring examples. The final trick is to propagate the multi-colour implication rules across all the possibles to prove colour identities within single possible colour maps which create new excluding pairs.

I was writing a paper on the finished theory with a worked example (anyone wants a copy email me) when several obvious conclusions hit me.

1) Colouring was a simple initial idea but it has become far too complicated because of all the bolt-on rules we had to add to make it work.

2) Most of the work is done by three very powerful principles:

a) Colouring of conjugate pairs
b) Iterative closure of the implication matrix
c) Proving colour contradictions so that we can assert colour pairs to be true/false
d) Proving colour identities so that we can discover new excluding pairs

I wondered if it would be possible to throw away everything except the above 4 ideas. So I tried to program it and it worked even better than I imagined. It ate up all the colouring examples with ease and solves many of the hardest published puzzles.

The new “supercoloring” approach uses matrix algebra to solve puzzles which have stalled using conventional methods and uses strict logic. It solves all of Rubylips’s “Nishio” puzzles without any suspect “what if” arguments. It’s very simple to program and very fast. I’ve not tried it yet but you could do it by hand on 81x81 grid paper 

I will outline the method and then show briefly how it solves Nishio 1.

1) Assign colours
Assign a different colour to every instance of every possible in the partial grid, ignoring certainties. This is quite different from simple colouring, which only colours conjugate pairs.

2) Create an exclusion matrix where excludes(u,v) = true if colour u excludes colour v somewhere on the partial grid. If we know that u=true then we can assert that v=false or if we know that u=true then we can assert that v=false. But if we only know that u=false we can assert nothing about v and if we only know that v=false we can assert nothing about u. This is quite different from simple colouring.

Formally, excludes(u,v) = not(u and v)

Colours exclude each other either if they sit as different possibles in the same cell or they are conjugate pairs of the same possible within a row, column or box. Note that the exclusion matrix is half symmetric, so that excludes(u,v) = excludes(v,u).

3) Create a conjugate matrix where conjugate(u,v) is true if colours u and v form a conjugate pair within a row, column or box.

Formally, conjugate(u,v) = (u and (not v)) or ((not u) and v)

The conjugate matrix is fully symmetrical - as in simple colouring – so that either (u=true and v=false) or (u=false and u=true). It is possible to compute the closure of this matrix by noting that if conjugate(u,v) and conjugate(v,w) then we can assert that u and w are identical and reduce the number of colours. This algorithm will iteratively determine the minimum number of conjugate colours and reduce the size of the matrices, but it is not necessary for the algorithm to complete. It’s a much better method that the clumsy ones I proposed originally for simple colouring.

4) Calculate an implication matrix where implies(u,v) is true if colour u implies v. If we know that u is true then v must also be true but if we know that u is false then we can assert nothing about v.

Formally, implies(u,v) = not(not(u) and v).

The implication matrix is easily initialised by setting implies(u,w) for every combination of u, v and w such that excludes(u,v) and conjugate(v,w).

5) Although the implication matrix is not symmetric, it is helpfully transitive. This enables us to compute the closure by iteratively setting implies(v,w) for every combination of colours u, v and w where implies(u,v) and implies(v,w). We now have everything we need to apply the two supercolouring rules.

6) The contradiction rule

If we can find a combination of colours u, v, w such that implies(u,v), implies(u,w) but excludes(v,w) then we have a contradiction which allows us to assert that u=false. This permits us to remove the possibles coloured u from everywhere on the grid. We can also assert that x=true for any colour x where conjugate(u,x), when enables us to force the possibles coloured x everywhere on the grid..

7) The identity rule

If we can prove that implies(u,v) and implies(v,u) where u and v are colours for the same possible p, the we can recolour the colour map for possible p by replacing every instance of colour v on the grid by colour u or vice-versa. We should of course do this consistently by selecting the lowest colour in every case. This may enable us to discover new excluding pairs.

That’s all there is to it … for the moment.

The puzzle “Nishio1” was one of 4 set by rubylips on Thu Apr 14 as follows: [updated 6-Aug to correct some errors in the transcript]

Code:
. . 6 ¦ . . . ¦ . . .
. 9 . ¦ . . . ¦ . 2 1
. 5 . ¦ . 3 8 ¦ . . .
------+-------+-------
3 . . ¦ . . . ¦ 7 . .
. . . ¦ 9 . 5 ¦ . . .
. . 1 ¦ . . . ¦ . . 6
------+-------+-------
. . . ¦ 2 4 . ¦ . 9 .
7 8 . ¦ . . . ¦ . 4 .
. . . ¦ . . . ¦ 3 . .


Conventional methods stall at the configuration:

Code:
148  14   6    ¦ 45   29   29   ¦ 58   3    7   
48   9    3    ¦ 7    56   46   ¦ 58   2    1   
2    5    7    ¦ 1    3    8    ¦ 4    6    9   
---------------+----------------+----------------
3    26   9    ¦ 68   1268 126  ¦ 7    5    4   
46   467  8    ¦ 9    67   5    ¦ 2    1    3   
5    27   1    ¦ 34   27   34   ¦ 9    8    6   
---------------+----------------+----------------
16   3    5    ¦ 2    4    7    ¦ 16   9    8   
7    8    2    ¦ 36   169  1369 ¦ 16   4    5   
9    16   4    ¦ 58   58   16   ¦ 3    7    2   


Supercolouring assigns the following colour map:

Code:
1a4a8a 1b4b   6 ¦ 4c5a 2a9a     2b9b     ¦ 5b8b 3 7
4c8b   9      3 ¦ 7    5b6a     4e6b     ¦ 5a8a 2 1
2      5      7 ¦ 1    3        8        ¦ 4    6 9
----------------+------------------------+----------
3      2c6c   9 ¦ 6d8e 1c2d6e8f 1d2a6f   ¦ 7    5 4
4b6g   4g6h7a 8 ¦ 9    6i7b     5        ¦ 2    1 3
5      2f7b   1 ¦ 3a4e 2c7a     3b4c     ¦ 9    8 6
----------------+------------------------+----------
1b6j   3      5 ¦ 2    4        7        ¦ 1a6g 9 8
7      8      2 ¦ 3b6l 1d6m9b   1h3a6n9a ¦ 1b6j 4 5
9      1a6g   4 ¦ 5b8f 5a8e     1b6j     ¦ 3    7 2


The reasoning leading to the first identity is shown in detail

Exclusion in {1,1} 1a excludes 8a
Conjugate in row 1 8a conjugates 8b
1a excludes 8a and 8a conjugates 8b so 1a implies 8b
Exclusion in {2,1} 4c excludes 8b
Conjugate in row 2 4c conjugates 4e
8b excludes 4c and 4c conjugates 4e so 8b implies 4e
1a implies 8b and 8b implies 4e so 1a implies 4e
Exclusion in {6,4} 3a excludes 4e
Conjugate in row 6 3a conjugates 3b
4e excludes 3a and 3a conjugates 3b so 4e implies 3b
Exclusion in {8,4} 3b excludes 6l
Conjugate in col 4 6d conjugates 6l
3b excludes 6l and 6l conjugates 6d so 3b implies 6d
Exclusion in {4,4} 6d excludes 8e
Conjugate in row 4 8e conjugates 8f
6d excludes 8e and 8e conjugates 8f so 6d implies 8f
3b implies 6d and 6d implies 8f so 3b implies 8f
Exclusion in {4,5} 1c excludes 8f
Conjugate in row 4 1c conjugates 1d
8f excludes 1c and 1c conjugates 1d so 8f implies 1d
3b implies 8f and 8f implies 1d so 3b implies 1d
4e implies 3b and 3b implies 1d so 4e implies 1d
1a implies 4e and 4e implies 1d so 1a implies 1d
Constraint in row 8 1d excludes 1b
Conjugate in row 1 1a conjugates 1b
1d excludes 1b and 1b conjugates 1a so 1d implies 1a
1a implies 1d and 1d implies 1a so 1a=1d
Recolour: {4,6} old=1d new=1a
Recolour: {8,5} old=1d new=1a

The reasoning leading to the first conflict is shown in detail

Exclusion in {1,2} 1b excludes 4b
Conjugate in row 5 4b conjugates 4g
1b excludes 4b and 4b conjugates 4g so 1b implies 4g
Exclusion in {5,2} 4g excludes 7a
Conjugate in row 5 7a conjugates 7b
4g excludes 7a and 7a conjugates 7b so 4g implies 7b
1b implies 4g and 4g implies 7b so 1b implies 7b
Exclusion in {6,2} 2f excludes 7b
Conjugate in row 6 2f conjugates 2c
7b excludes 2f and 2f conjugates 2c so 7b implies 2c
1b implies 7b and 7b implies 2c so 1b implies 2c
Constraint in row 4 2c excludes 2a
Conjugate in row 1 2a conjugates 2b
2c excludes 2a and 2a conjugates 2b so 2c implies 2b
Exclusion in {1,6} 2b excludes 9b
Conjugate in row 1 9a conjugates 9b
2b excludes 9b and 9b conjugates 9a so 2b implies 9a
Exclusion in {8,6} 3a excludes 9a
Conjugate in row 6 3a conjugates 3b
9a excludes 3a and 3a conjugates 3b so 9a implies 3b
2b implies 9a and 9a implies 3b so 2b implies 3b
2c implies 2b and 2b implies 3b so 2c implies 3b
Exclusion in {8,4} 3b excludes 6l
Conjugate in col 4 6d conjugates 6l
3b excludes 6l and 6l conjugates 6d so 3b implies 6d
2c implies 3b and 3b implies 6d so 2c implies 6d
Exclusion in {4,4} 6d excludes 8e
Conjugate in row 4 8e conjugates 8f
6d excludes 8e and 8e conjugates 8f so 6d implies 8f
2c implies 6d and 6d implies 8f so 2c implies 8f
1b implies 2c and 2c implies 8f so 1b implies 8f
Exclusion in {4,5} 1c excludes 8f
Conjugate in row 4 1c conjugates 1d
8f excludes 1c and 1c conjugates 1d so 8f implies 1d
Constraint in row 8 1d excludes 1b
Conjugate in row 1 1a conjugates 1b
1d excludes 1b and 1b conjugates 1a so 1d implies 1a
8f implies 1d and 1d implies 1a so 8f implies 1a
1b implies 8f and 8f implies 1a so 1b implies 1a
Constraint in row 8 1d excludes 1b
Conjugate in row 4 1c conjugates 1d
1b excludes 1d and 1d conjugates 1c so 1b implies 1c
Log fail: excludes 1 3
Conflict: 1b implies 1a and 1b implies 1c but 1a excludes 1c
Reduction: 1b=false in {9,6} before=16 after=6

AMcK


Last edited by AMcK on Sat Aug 06, 2005 8:45 am; edited 1 time in total
Back to top
View user's profile Send private message Send e-mail
angusj
Site Admin
Joined: 18 Jun 2005
Posts: 406
:

Items
PostPosted: Sat Jul 30, 2005 3:45 am    Post subject: Re: Nishio vs Supercolouring Reply with quote

AMcK wrote:
Apologies for the length and detail of this post but I suspect that I may have stumbled on a major breakthrough.

Hi Andrew.
I've just had a quickish look at this and therefore haven't fully grasped it all yet. However, provisionally I'd like to say ... WOW... since it looks so promising.

AMcK wrote:
Conflict: 1b implies 1a and 1b implies 1d but 1a excludes 1d

Could this contradiction be stated more simply by...
Conflict: 1b implies 1a but 1b is the conjugate of 1a
Back to top
View user's profile Send private message Visit poster's website
Nick70

Joined: 08 Jun 2005
Posts: 160
:

Items
PostPosted: Sat Jul 30, 2005 10:25 am    Post subject: Reply with quote

Thanks Andrew for the fresh look at coloring algorithms.

Of course your description is equivalent to the one I gave here, but I like the way you have formalized the final steps.

What I perhaps don't like is your use of three matrices (excludes[], conjugate[], implies[]) where I only use two (excludes[] and conjugate[]). I will give some careful thought to which of the two approaches is more efficient. I like excludes[] and conjugate[] because they are both symmetrical, while implies[] is not.
Implies[] is redundant, because it can always be derived from the other two. But that's not really important, because I use a set of rules which is equivalent to yours but doesn't need implies[] at all. This leads me to believe that my algorithm is more efficient.
Back to top
View user's profile Send private message
angusj
Site Admin
Joined: 18 Jun 2005
Posts: 406
:

Items
PostPosted: Sat Jul 30, 2005 11:00 am    Post subject: Reply with quote

Nick70 wrote:
Of course your description is equivalent to the one I gave here, but I like the way you have formalized the final steps.

Strange, I wonder how I missed your post???
Yes, it'd be great to exclude impies[] Very Happy.

Now, to get my head around this and see how I might incorporate this into SS ....
Back to top
View user's profile Send private message Visit poster's website
Nick70

Joined: 08 Jun 2005
Posts: 160
:

Items
PostPosted: Sat Jul 30, 2005 11:20 am    Post subject: Reply with quote

angusj wrote:
Now, to get my head around this and see how I might incorporate this into SS ....


You do realize that this means you'll have to allow coloring of pencilmarks, don't you? Smile
Back to top
View user's profile Send private message
angusj
Site Admin
Joined: 18 Jun 2005
Posts: 406
:

Items
PostPosted: Sat Jul 30, 2005 12:12 pm    Post subject: Reply with quote

Nick70 wrote:
You do realize that this means you'll have to allow coloring of pencilmarks, don't you? Smile

No, that's still most unlikely. However, I'd still like SS to easily determine when incorrect moves are made in these relatively 'obscure' puzzles.
Keep trying Very Happy.
Back to top
View user's profile Send private message Visit poster's website
AMcK

Joined: 07 Apr 2005
Posts: 89
:
Location: Cambridge

Items
PostPosted: Sun Jul 31, 2005 1:39 pm    Post subject: Reply with quote

Thanks for confirmation of consistency of the idea.
In its defence, the beauty of implies[i,j] is that computing the closure is that it's transitive so just a few lines of iterative code does an amazing amount of work.
I've now had to put logical backtrace from a log so that I can understand the complex logical reasoning. Example from Nishio2 below.
Interesting that it solves the Nishios Smile
AMcK
Code:
158   2     359   368   3568  58    4     7     19   
45    19    35    2347  2345  2457  19    8     6     
48    7     6     9     1     48    3     5     2     
9     3     25    17    28    17    6     4     58   
7     8     4     5     9     6     2     1     3     
25    6     1     2348  2348  248   58    9     7     
12    5     29    148   7     3     189   6     1489 
6     4     7     128   258   12589 1589  3     1589 
3     19    8     146   456   1459  7     2     1459 

Code:
1a5a8a     *          3a5b9a     3b6a8b     3c5c6b8c   5d8d       *          *          1b9b       
4a5e       1c9c       3d5f       2a3e4b7a   2b3f4c5g   2c4d5h7b   1d9d       *          *         
4e8e       *          *          *          *          4f8f       *          *          *         
*          *          2d5i       1e7c       2e8g       1f7d       *          *          5j8h       
*          *          *          *          *          *          *          *          *         
2f5k       *          *          2g3g4g8i   2h3h4h8j   2i4i8k     5l8l       *          *         
1g2j       *          2k9e       1h4j8m     *          *          1i8n9f     *          1j4k8o9g   
*          *          *          1k2l8p     2m5m8q     1l2n5n8r9h 1m5o8s9i   *          1n5p8t9j   
*          1o9k       *          1p4l6c     4m5q6d     1q4n5r9l   *          *          1r4o5s9m   

First conflict that enables a reduction
Code:
Constraint in row 7 1b excludes 1i
Conjugate in row 1 1a conjugates 1b
1i excludes 1b and 1b conjugates 1a so 1i implies 1a
Constraint in col 7 1a excludes 1i
Conjugate in row 1 1a conjugates 1b
1i excludes 1a and 1a conjugates 1b so 1i implies 1b
Constraint in row 1 1a excludes 1b
Conflict: 1i implies 1a and 1i implies 1b but 1a excludes 1b
Reduction: 1i=false in {7,7} before=189 after=89
Back to top
View user's profile Send private message Send e-mail
Nick70

Joined: 08 Jun 2005
Posts: 160
:

Items
PostPosted: Sun Jul 31, 2005 3:31 pm    Post subject: Reply with quote

angusj wrote:
No, that's still most unlikely. However, I'd still like SS to easily determine when incorrect moves are made in these relatively 'obscure' puzzles.


I don't understand what you mean, surely you have a backtracking solver integrated in the program, so you can find the solution of any puzzle? If not--well, a backtracking solver is orders of magnitude simpler than a "logic" solver, so there's no reason not to write one Smile
Back to top
View user's profile Send private message
angusj
Site Admin
Joined: 18 Jun 2005
Posts: 406
:

Items
PostPosted: Sun Jul 31, 2005 10:38 pm    Post subject: Reply with quote

Nick70 wrote:
I don't understand what you mean, surely you have a backtracking solver integrated in the program, so you can find the solution of any puzzle?

How can you backtrack if you don't know the final grid?
I haven't yet gotten around to constructing an algorithm to work out solutions for puzzles which don't have a known 'logical' solution. (See my other post regarding this in the 'Software' section.)
Back to top
View user's profile Send private message Visit poster's website
Nick70

Joined: 08 Jun 2005
Posts: 160
:

Items
PostPosted: Sun Jul 31, 2005 10:52 pm    Post subject: Reply with quote

angusj wrote:
How can you backtrack if you don't know the final grid?
I haven't yet gotten around to constructing an algorithm to work out solutions for puzzles which don't have a known 'logical' solution.


That's weird, since it's so much easier than finding 'logical' steps.

When I say 'backtrack' I mean that the program, after doing all single candidate moves, just picks a cell or a unit with multiple possibilities (usually two) and tries them all, recursively. You can either stop at the first solution found, or at the second (if you just need to know if the solution is unique or not), or continue running until all possibilities are evaluated (if you need an exact count of solutions--of course this can take a LONG time if there are millions of solutions).

Knuth's paper on dancing links is a must read. He also provides sample code though it isn't very easy to understand since it's designed for speed rather than readability (it avoid recursion using gotos).
Back to top
View user's profile Send private message
angusj
Site Admin
Joined: 18 Jun 2005
Posts: 406
:

Items
PostPosted: Sun Jul 31, 2005 11:37 pm    Post subject: Reply with quote

Nick70 wrote:
That's weird, since it's so much easier than finding 'logical' steps.

I know it's not hard, it just hasn't been a priority so far. Simple Sudoku started as a 'solver', so my focus was on finding logical solutions.

Nick70 wrote:
When I say 'backtrack' ...

I really do understand how I can recursively derive a solution or solutions, it's just that it hasn't been a priority so far. (Perhaps when I said 'algorithm' above, I should have said 'code' - since I do already know the conceptual steps required.)

Anyhow, I do appreciate all your suggestions.
Back to top
View user's profile Send private message Visit poster's website
AMcK

Joined: 07 Apr 2005
Posts: 89
:
Location: Cambridge

Items
PostPosted: Sun Jul 31, 2005 11:55 pm    Post subject: Reply with quote

Here's the pseudo code that explains how I adapted my logical solver to do brute force - and also multiple solution counting.

Code:
bruteforce(g)
  h=copy(g)
  logicallysolve(g)
  if solved(g) then
   return true
  else if insoluble(g) then
    return false
  else
   for each cell in g with possibles
    for each possible in cell
      cell=possible
      if bruteforce(h) then
        g=h
        return true
      else
        h=copy(g)
      end if
    next possible
  next cell
end if

The use of the conventional solver helps speed up the algorithm significantly althugh I find the optimum is to restrict the subset/superset rules to about order 4.
Multiple solution counting is done by omitting the return true when a correct solution is found and just replacing this with an incerementer/saver.
As Nick70 points out, there's an opportunity for Knuth's dancing links to optimise the copying/restoring of the grid.
Back to top
View user's profile Send private message Send e-mail
Nick70

Joined: 08 Jun 2005
Posts: 160
:

Items
PostPosted: Mon Aug 01, 2005 7:08 am    Post subject: Reply with quote

[quote="AMcK"]
Code:
bruteforce(g)
   for each cell in g with possibles
    for each possible in cell


That's wrong. You don't need to do the recursion on EVERY cell. Just pick one.
Back to top
View user's profile Send private message
Nick70

Joined: 08 Jun 2005
Posts: 160
:

Items
PostPosted: Mon Aug 01, 2005 7:22 am    Post subject: Reply with quote

AMcK wrote:
In its defence, the beauty of implies[i,j] is that computing the closure is that it's transitive so just a few lines of iterative code does an amazing amount of work.


Previously I said that with my algorithm I only need excludes[] and conjugate[]. This isn't quite correct.

I only need excludes[], actually. conjugate[] is implicit in the number assigned to the pencilmark. You are assigning numbers at random, while during the initial coloring I assign conjugate numbers to conjugate pencilmarks. Always. There is no difference whatsoever between two possibles in a cell or two positions in a unit.

So the conjugate[] matrix isn't needed. It would be just conjugate(x,-x)=1 and the rest 0.

excludes[] is the only matrix I build.

implies[]--what would it be? Well as you say,
implies(x,y) iff excludes(x,z) and conjugate(z,y).
In my terms, you just have that
implies(x,y)=excludes(x,-y).

You do the closure of implies[]:
if implies(x,y) and implies(y,z) then implies(x,z).
I just do the closure of excludes[]:
if excludes(x,-y) and excludes(y,-z) then excludes(x,-z). [edited to fix a typo]

Finally the recoloring: you do
if implies(x,y) and implies(y,x) then x=y
I do
if excludes(x,-y) and excludes(y,-x) then x=y


Last edited by Nick70 on Mon Aug 01, 2005 9:04 am; edited 1 time in total
Back to top
View user's profile Send private message
angusj
Site Admin
Joined: 18 Jun 2005
Posts: 406
:

Items
PostPosted: Mon Aug 01, 2005 8:48 am    Post subject: Reply with quote

Nick70 wrote:
You do the closure of implies[]:
if implies(x,y) and implies(y,z) then implies(x,z).
I just do the closure of excludes[]:
if excludes(x,-y) and excludes(y,z) then excludes(x,z).

I'm probably in over my head here but ...
aren't you assuming here that implies(y,z) is the same as excludes(-y,z) which of course may not be true?
Shouldn't this be written:
if excludes(x,-y) and excludes(y,-z) then excludes(x,-z).
Anyhow, this is much harder to understand though it may be mathematically/programmatically simpler.

Edit: By the way Nick, I've found a proper "hidden quad" for you, I'm just waiting for Pappocom's forum to get back online.


Last edited by angusj on Mon Aug 01, 2005 9:16 am; edited 2 times in total
Back to top
View user's profile Send private message Visit poster's website
Display posts from previous:   
Post new topic   Reply to topic    Sudoku Programmers Forum Index -> Solving sudoku All times are GMT
Goto page 1, 2, 3  Next
Page 1 of 3

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
Sudoku Programmers topic RSS feed 


Powered by phpBB © 2001, 2005 phpBB Group

Igloo Theme Version 1.0 :: Created By: Andrew Charron